fix: expose caserun trace lookup hints
This commit is contained in:
@@ -233,7 +233,9 @@ bun tools/hwlab-cli/bin/hwlab-cli.ts case run result <runId>
|
||||
bun tools/hwlab-cli/bin/hwlab-cli.ts case run logs <runId> --tail 8000
|
||||
```
|
||||
|
||||
`case run` 在当前版本只负责编排流程和记录原始证据,不做 evidence 自动评价、自动打分或门禁判断。流程是:prepare 先在 subject repo 本地 checkout 下创建 `.worktree/caserun-<runId>` 隔离 worktree并写入 run-local spec,然后生成 Code Agent prompt/context,创建或调用 HWLAB Code Agent session,把 isolated worktree、run-local HWPOD spec、任务目标和约束交给 Code Agent;agent 阶段结束或超时后,CaseRun 采集 agent trace/terminal state/commandExecution 摘要、subject worktree 的 `git status` / `git diff --stat` / `git diff --binary`,再按 case 定义执行 runner 后置 `hwpod-cli build` 和 `cmd.run` Keil 异步 job-status 查询,最后把原始证据和可读归档写入 case registry repo 的 `runs/<caseId>/<runId>/`。固定归档包括 `evidence.json`、`summary.md`、`run.json`、`result.json`、`agent-prompt.md`、`agent-diff.patch`、run-local `.hwlab/hwpod-spec.yaml`、worker stdout/stderr、`artifact-manifest.json`,以及共享 Web/CLI trace renderer 生成的 `agent-messages.json`、`agent-trace.md`、`agent-transcript.md` 和 `final-response.md`。`agent-trace.md` / `agent-transcript.md` 必须包含 rendered trace rows、renderer 标识和 subject diff 区块;`final-response.md` 在存在权威 finalResponse 时保存正文,不存在时显式记录 `finalResponse=null`、terminal/error rows 和缺失原因。下载不作为 compile-only CaseRun 的隐式步骤;后续需要下载时,应作为独立 action 显式加入 case 定义和流程记录。
|
||||
`case run` 在当前版本只负责编排流程和记录原始证据,不做 evidence 自动评价、自动打分或门禁判断。流程是:prepare 先在 subject repo 本地 checkout 下创建 `.worktree/caserun-<runId>` 隔离 worktree并写入 run-local spec,然后生成 Code Agent prompt/context,创建或调用 HWLAB Code Agent session,把 isolated worktree、run-local HWPOD spec、任务目标和约束交给 Code Agent;agent 阶段结束或超时后,CaseRun 记录 agent identity、terminal result 里已有的终态字段、trace lookup hint、subject worktree 的 `git status` / `git diff --stat` / `git diff --binary`,再按 case 定义执行 runner 后置 `hwpod-cli build` 和 `cmd.run` Keil 异步 job-status 查询,最后把原始证据和可读归档写入 case registry repo 的 `runs/<caseId>/<runId>/`。固定归档包括 `evidence.json`、`summary.md`、`run.json`、`result.json`、`agent-prompt.md`、`agent-diff.patch`、run-local `.hwlab/hwpod-spec.yaml`、worker stdout/stderr、`artifact-manifest.json`,以及 `agent-messages.json`、`agent-trace.md`、`agent-transcript.md` 和 `final-response.md`。`agent-trace.md` / `agent-transcript.md` 必须包含 trace lookup hint、renderer 标识和 subject diff 区块;如果归档了完整 rendered rows,其语义必须来自共享 Web/CLI trace renderer,不能由 CaseRun 自行发明。`final-response.md` 在 terminal result 存在权威 finalResponse 时保存正文,不存在时显式记录 `finalResponse=null`、terminal/error rows 和缺失原因。下载不作为 compile-only CaseRun 的隐式步骤;后续需要下载时,应作为独立 action 显式加入 case 定义和流程记录。
|
||||
|
||||
CaseRun 的 trace 处理必须遵循 [spec-v02-code-agent-trace.md](spec-v02-code-agent-trace.md) 的组合优先原则:CaseRun 只负责稳定返回和归档 `traceId`、`sessionId`、`conversationId`、`threadId`、AgentRun 执行线索以及 `traceLookup.commands.*`,完整 trace/result/session 读取继续使用已有 `hwlab-cli client agent trace/result/inspect`。CaseRun 不应在自身内部再实现一套 `/v1/agent/chat/trace/{traceId}` 查询器、row renderer、session 反查或 final response 归因逻辑;如果已有 agent CLI 的 trace 可见性不足,先改进 `client agent trace/result/inspect`,再让 CaseRun 输出新的 ID 或 hint。归档中的 `agent-trace.json` 可以是 `lookupOnly=true` 的 identity manifest,表示“用这些 ID 调既有 CLI 查完整 trace”,不能伪装成完整 trace dump。
|
||||
|
||||
长时间 CaseRun 必须优先使用 `case run start/status/result/logs` 短连接形态,而不是用 shell/nohup 包装同步 `case run`。`start` 只负责创建 `.state/hwlab-cli/caserun/<runId>/`、启动后台 worker 并立即返回 `runId`、PID、stdout/stderr 路径和下一条查询命令;`status`、`result`、`logs` 都是短查询。`run.json` 是阶段状态权威,阶段开始时就要刷新 `status`、`stage`、`traceId/sessionId/conversationId/threadId`、AgentRun/trace 轮询信息、stdout/stderr byte count 和下一条轮询命令;长等待期间不得出现无输出、无 trace、无状态文件更新的黑洞窗口。
|
||||
|
||||
|
||||
@@ -100,6 +100,21 @@ Web、CLI 和 CI 的一致性按“入口同源、API 同 path、renderer 同代
|
||||
|
||||
如果 Web、CLI 或 CI 只覆盖了上述链路的一段,不能称为同路径修复。例如只修 Web DOM、只修 CLI formatter、只修 session cache、只修 AgentRun events filter、或只让 CI mock helper 变绿,都可能继续保留另一条 final/trace 路径,导致“修过很多次但现象还在”。Trace 相关变更必须同时对齐 authority、API、renderer 和 component plan 四层中的相关部分。
|
||||
|
||||
## 组合优先原则
|
||||
|
||||
CaseRun、harness、runner、诊断脚本和其他上层编排工具需要查看 Code Agent trace 时,必须采用“ID + 既有工具组合”的方式,而不是在上层工具里重新实现 trace/result/session 查询和渲染逻辑。上层工具的职责是稳定返回 `traceId`、`sessionId`、`conversationId`、`threadId`、AgentRun `runId/commandId` 线索和可复制的下一条 HWLAB CLI 命令;完整 trace、result、inspect 和 Web renderer 语义继续由 `hwlab-cli client agent trace/result/inspect` 提供。
|
||||
|
||||
推荐输出合同:
|
||||
|
||||
- `traceLookup.source=hwlab-cli.client.agent`,说明后续查询复用既有 HWLAB Agent CLI。
|
||||
- `traceLookup.strategy=id_plus_existing_cli`,说明当前工具只提供 identity 和组合 hint。
|
||||
- `traceLookup.commands.result=hwlab-cli client agent result <traceId> --base-url <web-origin>`。
|
||||
- `traceLookup.commands.trace=hwlab-cli client agent trace <traceId> --base-url <web-origin> --render web`。
|
||||
- `traceLookup.commands.inspect=hwlab-cli client agent inspect --trace-id <traceId> --base-url <web-origin>`。
|
||||
- 有 `sessionId` 时可以补 `traceLookup.commands.sessionStatus=hwlab-cli client agent session status <sessionId> --base-url <web-origin>`。
|
||||
|
||||
这条规则的目的是通过组合复用保持单一 trace authority:CaseRun 或 harness 可以归档 `traceLookup` 和由 result payload 直接可得的终态摘要,但不得为了方便再发明一套 `/trace/{traceId}` fetch、row renderer、session 反查、final response 归因或 AgentRun events 过滤器。需要更多可见性时,先改进 `hwlab-cli client agent trace/result/inspect` 这些已有入口,再让上层工具返回新的 hint 或消费这些入口的稳定字段。
|
||||
|
||||
## 派生缓存边界
|
||||
|
||||
允许存在的派生状态只有两个用途:降低轮询开销和提升 UI 可见性。派生状态必须可被权威路径重建,且不得成为 terminal result 的替代 authority。
|
||||
|
||||
@@ -328,22 +328,22 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit
|
||||
assert.equal(result.payload.agent.stageStatus, "completed");
|
||||
assert.equal(result.payload.agent.traceId.startsWith("trc_case_"), true);
|
||||
assert.equal(result.payload.agent.timeoutMs, 234000);
|
||||
assert.ok(result.payload.agentTrace.hwpodBuildCommandCount >= 1);
|
||||
assert.ok(result.payload.evidence.agentTrace.commandCount > 0);
|
||||
assert.ok(result.payload.evidence.agentTrace.commandCount <= 30);
|
||||
assert.ok(result.payload.evidence.agentTrace.hwpodCommandCount >= 3);
|
||||
assert.equal(result.payload.evidence.agentTrace.keilJobCandidates.includes("20260605_203835_798515c0"), true);
|
||||
assert.equal(result.payload.traceLookup.commands.trace, `hwlab-cli client agent trace ${result.payload.agent.traceId} --base-url http://web.test --render web`);
|
||||
assert.equal(result.payload.traceLookup.commands.result, `hwlab-cli client agent result ${result.payload.agent.traceId} --base-url http://web.test`);
|
||||
assert.equal(result.payload.traceLookup.commands.inspect, `hwlab-cli client agent inspect --trace-id ${result.payload.agent.traceId} --base-url http://web.test`);
|
||||
assert.equal(result.payload.agentTrace.source, "caserun-identity");
|
||||
assert.equal(result.payload.agentTrace.lookupOnly, true);
|
||||
assert.equal(result.payload.agentTrace.hwpodBuildCommandCount, 0);
|
||||
assert.equal(result.payload.evidence.agentTrace.commandCount, 0);
|
||||
assert.equal(result.payload.evidence.agentTrace.hwpodCommandCount, 0);
|
||||
assert.equal(result.payload.evidence.agentTrace.keilJobCandidates.includes("20260605_203835_798515c0"), false);
|
||||
assert.equal(result.payload.evidence.agentTask.timeoutMs, 234000);
|
||||
assert.equal(result.payload.agent.polls <= 3, true);
|
||||
assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod-ctl spec validate --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true);
|
||||
assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod inspect --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true);
|
||||
assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod build --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true);
|
||||
assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /cat .*hwpod-cli\/SKILL\.md/u.test(item.command ?? "") && /hwpod build/u.test(item.bodyPreview ?? "")), true);
|
||||
assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod build --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true);
|
||||
assert.equal(result.payload.evidence.hwpod.source, "case-run-runner-post-agent-compile-check");
|
||||
assert.match(result.payload.agentDiff.statusShort, /projects\/01_baseline\/main\.c/u);
|
||||
assert.equal(result.payload.build.autoEvaluation, false);
|
||||
assert.ok(result.payload.build.agentTraceHwpodCommandCount >= 3);
|
||||
assert.equal(result.payload.build.traceLookup.commands.trace, result.payload.traceLookup.commands.trace);
|
||||
assert.equal(result.payload.build.agentTraceHwpodCommandCount, 0);
|
||||
assert.equal(result.payload.build.hwpodExitCode, 0);
|
||||
assert.equal(result.payload.build.hwpodSource, "case-run-runner-post-agent-compile-check");
|
||||
assert.equal(result.payload.build.keilStatus, "completed");
|
||||
@@ -374,7 +374,10 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit
|
||||
assert.equal(manifest.trace.traceId, result.payload.agent.traceId);
|
||||
assert.equal(manifest.trace.sessionId, "ses_case_run");
|
||||
assert.equal(manifest.trace.conversationId, "cnv_case_d601-f103-v2-compile_run-agent-flow");
|
||||
assert.equal(manifest.trace.agentRun.runId, "run_agent_trace");
|
||||
assert.equal(manifest.trace.source, "caserun-identity");
|
||||
assert.equal(manifest.trace.lookupOnly, true);
|
||||
assert.equal(manifest.trace.lookup.commands.trace, result.payload.traceLookup.commands.trace);
|
||||
assert.equal(manifest.traceLookup.commands.inspect, result.payload.traceLookup.commands.inspect);
|
||||
assert.equal(manifest.prompt.sha256, result.payload.evidence.agent.promptSha256);
|
||||
assert.equal(manifest.subject.commitId, SUBJECT_COMMIT_ID);
|
||||
assert.equal(manifest.diff.sha256, result.payload.evidence.agentDiff.diffPatchSha256);
|
||||
@@ -385,10 +388,8 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit
|
||||
assert.equal(manifest.readableAgent.finalResponsePath, "final-response.md");
|
||||
assert.equal(manifest.readableAgent.finalResponse.present, true);
|
||||
assert.equal(manifest.readableAgent.finalResponse.text, "done");
|
||||
assert.deepEqual(manifest.agentStage.kinds, ["spec-validate", "inspect", "workspace-edit", "build", "download", "uart-read"]);
|
||||
assert.deepEqual(manifest.agentStage.kinds, []);
|
||||
assert.equal(manifest.agentStage.autoEvaluation, false);
|
||||
assert.equal(manifest.agentStage.commands.some((item: any) => item.kind === "download" && /uvoptx file not found/u.test(item.detail)), true);
|
||||
assert.equal(manifest.agentStage.commands.some((item: any) => item.kind === "uart-read" && /hwpod_uart_monitor_not_active/u.test(item.detail)), true);
|
||||
assert.equal(manifest.decisions.autoEvaluation, false);
|
||||
assert.equal(manifest.decisions.downloadSkipped, undefined);
|
||||
assert.equal(manifest.decisions.runnerPostAgentCompileCheck, "recorded");
|
||||
@@ -399,10 +400,14 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit
|
||||
assert.equal(manifest.files.some((item: any) => item.path === ".hwlab/hwpod-spec.yaml"), true);
|
||||
const messages = JSON.parse(await readFile(path.join(registryRunDir, "agent-messages.json"), "utf8"));
|
||||
assert.equal(messages.renderer, "tools/src/hwlab-cli/trace-renderer:traceDisplayRows");
|
||||
assert.equal(messages.lookupOnly, true);
|
||||
assert.equal(messages.traceLookup.commands.trace, result.payload.traceLookup.commands.trace);
|
||||
assert.equal(messages.finalResponse.present, true);
|
||||
assert.match(JSON.stringify(messages.rows), /hwpod build --spec \.hwlab\/hwpod-spec\.yaml/u);
|
||||
assert.equal(messages.rows.length, 0);
|
||||
const traceMarkdown = await readFile(path.join(registryRunDir, "agent-trace.md"), "utf8");
|
||||
assert.match(traceMarkdown, /renderer: tools\/src\/hwlab-cli\/trace-renderer:traceDisplayRows/u);
|
||||
assert.match(traceMarkdown, /traceLookupStrategy: id_plus_existing_cli/u);
|
||||
assert.match(traceMarkdown, /client agent trace trc_case_/u);
|
||||
assert.match(traceMarkdown, /## Subject Diff/u);
|
||||
assert.match(traceMarkdown, /\+printf\("hello"\);/u);
|
||||
const finalResponse = await readFile(path.join(registryRunDir, "final-response.md"), "utf8");
|
||||
@@ -410,16 +415,13 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit
|
||||
assert.match(finalResponse, /done/u);
|
||||
assert.equal(result.payload.collect.caseRepoRunDir, registryRunDir);
|
||||
assert.equal(result.payload.collect.artifactManifestPath, path.join(registryRunDir, "artifact-manifest.json"));
|
||||
assert.equal(result.payload.collect.agentStage.commands.some((item: any) => item.kind === "download"), true);
|
||||
assert.deepEqual(result.payload.collect.agentStage.commands, []);
|
||||
assert.equal(result.payload.collect.caseRepoFiles.some((item: string) => item.endsWith("artifact-manifest.json")), true);
|
||||
const registryResult = JSON.parse(await readFile(path.join(registryRunDir, "result.json"), "utf8"));
|
||||
assert.equal(registryResult.collect.artifactManifestPath, path.join(registryRunDir, "artifact-manifest.json"));
|
||||
const summaryMarkdown = await readFile(path.join(registryRunDir, "summary.md"), "utf8");
|
||||
assert.match(summaryMarkdown, /## Agent HWPOD Raw Steps/u);
|
||||
assert.match(summaryMarkdown, /download/u);
|
||||
assert.match(summaryMarkdown, /uvoptx file not found/u);
|
||||
assert.match(summaryMarkdown, /uart-read/u);
|
||||
assert.match(summaryMarkdown, /hwpod_uart_monitor_not_active/u);
|
||||
assert.match(summaryMarkdown, /traceLookupStrategy: id_plus_existing_cli/u);
|
||||
assert.match(summaryMarkdown, /traceCommand: hwlab-cli client agent trace trc_case_/u);
|
||||
assert.doesNotMatch(summaryMarkdown, /downloadSkipped/u);
|
||||
|
||||
const prompt = await readFile(path.join(cwd, ".state", "hwlab-cli", "caserun", "run-agent-flow", "agent-prompt.md"), "utf8");
|
||||
@@ -444,7 +446,7 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit
|
||||
assert.match(agentChatRequest.body.workspaceFiles[0].content, /HWLAB-CASE-F103.*caserun-run-agent-flow/u);
|
||||
assert.doesNotMatch(agentChatRequest.body.message, /path: "F:\\Work\\HWLAB-CASE-F103\\\.worktree\\caserun-run-agent-flow"/u);
|
||||
assert.equal(requests.some((item) => item.url === "http://web.test/v1/agent/chat/result/" + result.payload.agent.traceId), true);
|
||||
assert.equal(requests.some((item) => item.url === "http://web.test/v1/agent/chat/trace/" + result.payload.agent.traceId), true);
|
||||
assert.equal(requests.some((item) => item.url === "http://web.test/v1/agent/chat/trace/" + result.payload.agent.traceId), false);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -587,6 +589,7 @@ test("case run status reports latest run stage when async control stage is stale
|
||||
assert.equal(result.exitCode, 0);
|
||||
assert.equal(result.payload.ready, false);
|
||||
assert.equal(result.payload.stage, "agent-diff-collected");
|
||||
assert.equal(result.payload.traceLookup.commands.trace, "hwlab-cli client agent trace trc_case --base-url http://web.test --render web");
|
||||
assert.equal(result.payload.staleControlWarning.code, "stale_control_stage");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
@@ -680,10 +683,6 @@ function caseRunFlowFetch(requests: any[]) {
|
||||
resultPolls += 1;
|
||||
return new Response(JSON.stringify({ ok: true, status: resultPolls > 1 ? "completed" : "running", reply: { content: "done" } }), { status: 200, headers: { "content-type": "application/json" } });
|
||||
}
|
||||
if (String(url).startsWith("http://web.test/v1/agent/chat/trace/")) {
|
||||
const events = caseRunTraceEvents();
|
||||
return new Response(JSON.stringify({ ok: true, status: "completed", sourceEventCount: events.length, renderedRowCount: 0, traceSummary: { terminalStatus: "completed", agentRun: { runId: "run_agent_trace", commandId: "cmd_agent_trace" } }, events }), { status: 200, headers: { "content-type": "application/json" } });
|
||||
}
|
||||
if (body.ops?.[0]?.args?.argv?.includes("status") && body.ops?.length === 1) {
|
||||
return hwpodResponse([{ exitCode: 0, stdout: body.ops[0].args.argv.includes("--short") ? " M projects/01_baseline/main.c\n" : "" }]);
|
||||
}
|
||||
@@ -700,23 +699,6 @@ function caseRunFlowFetch(requests: any[]) {
|
||||
};
|
||||
}
|
||||
|
||||
function caseRunTraceEvents() {
|
||||
const events: any[] = [
|
||||
{ seq: 1, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "cat .agents/skills/hwpod-cli/SKILL.md", message: "use `hwpod build` for compile", exitCode: 0 },
|
||||
{ seq: 248, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod-ctl spec validate --spec .hwlab/hwpod-spec.yaml'", message: "validated", exitCode: 0 },
|
||||
{ seq: 323, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod inspect --spec .hwlab/hwpod-spec.yaml'", message: "inspected", exitCode: 0 },
|
||||
{ seq: 360, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod workspace insert-after --spec .hwlab/hwpod-spec.yaml --path projects/01_baseline/User/main.c --anchor ready --line marker'", message: "inserted marker", exitCode: 0 },
|
||||
{ seq: 480, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod download --spec .hwlab/hwpod-spec.yaml'", message: "uvoptx file not found; program accepted job 20260605_213000_aaaaaaaa", exitCode: 1 },
|
||||
{ seq: 520, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod uart read --spec .hwlab/hwpod-spec.yaml --port uart1 --max-bytes 4096'", message: "hwpod_uart_monitor_not_active on COM9", exitCode: 1 }
|
||||
];
|
||||
for (let index = 0; index < 31; index += 1) {
|
||||
const seq = 400 + index * 2;
|
||||
events.push({ seq, label: "item/commandExecution:started", type: "tool_call", toolName: "commandExecution", status: "started", command: "/bin/sh -lc 'hwpod build --spec .hwlab/hwpod-spec.yaml'", message: "commandExecution inProgress: /bin/sh -lc 'hwpod build --spec .hwlab/hwpod-spec.yaml'" });
|
||||
events.push({ seq: seq + 1, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod build --spec .hwlab/hwpod-spec.yaml'", message: index === 0 ? "{\"job_id\":\"20260605_203835_798515c0\"}" : "build completed", exitCode: 0 });
|
||||
}
|
||||
return events;
|
||||
}
|
||||
|
||||
function hwpodResponse(outputs: Array<{ exitCode: number; stdout?: string; stderr?: string }>) {
|
||||
return new Response(JSON.stringify({ body: { results: outputs.map((output) => ({ ok: output.exitCode === 0, output })) } }), { status: 200, headers: { "content-type": "application/json" } });
|
||||
}
|
||||
|
||||
+119
-16
@@ -105,6 +105,10 @@ type AgentRunStage = {
|
||||
lastPollAt?: string;
|
||||
lastHttpStatus?: number;
|
||||
nextPollCommand?: string;
|
||||
traceCommand?: string;
|
||||
inspectCommand?: string;
|
||||
sessionCommand?: string;
|
||||
traceLookup?: Record<string, unknown>;
|
||||
sessionResponse: Record<string, unknown> | null;
|
||||
error?: Record<string, unknown> | null;
|
||||
};
|
||||
@@ -135,6 +139,9 @@ type AgentTraceStage = {
|
||||
traceId: string;
|
||||
status: string;
|
||||
httpStatus: number;
|
||||
source?: string;
|
||||
lookupOnly?: boolean;
|
||||
lookup?: Record<string, unknown>;
|
||||
sourceEventCount?: number;
|
||||
renderedRowCount?: number;
|
||||
commandCount: number;
|
||||
@@ -246,6 +253,7 @@ export async function runCaseRun(context: CaseContext) {
|
||||
prepare: prepared.summary,
|
||||
agent: agentSummary(agentStage.agent),
|
||||
agentTrace: traceSummary(traceStage.trace),
|
||||
traceLookup: agentTraceLookupForRun(traceStage.run),
|
||||
agentDiff: diffSummary(diffStage.diff),
|
||||
build: built.summary,
|
||||
collect: finalCollect,
|
||||
@@ -358,6 +366,7 @@ export async function statusCaseRun(context: CaseContext) {
|
||||
registryArchive: registryArchive?.summary ?? null,
|
||||
prepare: run.subject?.worktreePath ? prepareSummary(run) : null,
|
||||
agent: run.agent ? agentSummary(run.agent) : null,
|
||||
traceLookup: run.agent ? agentTraceLookupForRun(run) : null,
|
||||
agentDiff: run.agentDiff ? diffSummary(run.agentDiff) : null,
|
||||
evidencePath: text(run.evidencePath ?? control.resultPath) || null,
|
||||
nextPollCommand: `hwlab-cli case run status ${run.runId} --state-dir ${stateRoot(context)}`,
|
||||
@@ -381,6 +390,8 @@ export async function resultCaseRun(context: CaseContext) {
|
||||
stage: stage.stage,
|
||||
staleControlWarning: stage.warning,
|
||||
ready: false,
|
||||
agent: run.agent ? agentSummary(run.agent) : null,
|
||||
traceLookup: run.agent ? agentTraceLookupForRun(run) : null,
|
||||
nextPollCommand: `hwlab-cli case run status ${run.runId} --state-dir ${stateRoot(context)}`,
|
||||
logsCommand: `hwlab-cli case run logs ${run.runId} --state-dir ${stateRoot(context)} --tail 8000`
|
||||
}, "running");
|
||||
@@ -406,6 +417,7 @@ export async function resultCaseRun(context: CaseContext) {
|
||||
registryArchive: registryArchive?.summary ?? null,
|
||||
summary: evidence ? buildSummary(evidence) : null,
|
||||
agent: run.agent ? agentSummary(run.agent) : null,
|
||||
traceLookup: run.agent ? agentTraceLookupForRun(run) : null,
|
||||
agentDiff: run.agentDiff ? diffSummary(run.agentDiff) : null,
|
||||
evidence: context.parsed.full === true ? evidence : compactObject(evidence),
|
||||
result: context.parsed.full === true ? result : compactObject(result)
|
||||
@@ -503,6 +515,7 @@ export async function buildCaseRun(context: CaseContext, prepared?: PreparedCase
|
||||
subject: run.subject,
|
||||
agentTask: run.agentTask ? agentTaskSummary(run.agentTask) : null,
|
||||
agent: run.agent ? agentSummary(run.agent) : null,
|
||||
traceLookup: agentTraceLookupForRun(run),
|
||||
agentTrace: run.agentTrace ? traceSummary(run.agentTrace) : null,
|
||||
agentDiff: run.agentDiff ? diffSummary(run.agentDiff) : null,
|
||||
hwpod: clean({
|
||||
@@ -543,6 +556,7 @@ export async function collectCaseRun(context: CaseContext, prepared?: PreparedCa
|
||||
caseRepoFiles: [],
|
||||
artifactManifestPath: null,
|
||||
trace: null,
|
||||
traceLookup: agentTraceLookupForRun(run, evidence),
|
||||
status: evidence.status,
|
||||
autoEvaluation: false
|
||||
};
|
||||
@@ -684,6 +698,7 @@ async function runAgentTaskStage(context: CaseContext, run: PreparedCaseRun) {
|
||||
}
|
||||
const resultPath = text(accepted.body?.resultUrl) || `/v1/agent/chat/result/${encodeURIComponent(traceId)}`;
|
||||
const resultUrl = resultPath.startsWith("http") ? resultPath : `${baseUrl}${resultPath}`;
|
||||
const traceLookup = agentTraceLookup({ baseUrl, traceId, sessionId, conversationId, threadId });
|
||||
const submittedAgent: AgentRunStage = clean({
|
||||
stageStatus: "submitted",
|
||||
baseUrl,
|
||||
@@ -702,7 +717,11 @@ async function runAgentTaskStage(context: CaseContext, run: PreparedCaseRun) {
|
||||
timeoutMs: effectiveAgentTimeoutMs(context, run.agentTask),
|
||||
pollIntervalMs: effectiveAgentPollIntervalMs(context, run.agentTask),
|
||||
resultUrl,
|
||||
nextPollCommand: `hwlab-cli client agent result ${traceId} --base-url ${baseUrl}`,
|
||||
nextPollCommand: traceLookup.commands.result,
|
||||
traceCommand: traceLookup.commands.trace,
|
||||
inspectCommand: traceLookup.commands.inspect,
|
||||
sessionCommand: traceLookup.commands.sessionStatus,
|
||||
traceLookup,
|
||||
sessionResponse: compactObject(session.body)
|
||||
});
|
||||
run = await updateRun(context, run, { stage: "agent-running", agent: submittedAgent });
|
||||
@@ -727,7 +746,11 @@ async function runAgentTaskStage(context: CaseContext, run: PreparedCaseRun) {
|
||||
resultUrl,
|
||||
lastPollAt: result.lastPollAt,
|
||||
lastHttpStatus: result.lastHttpStatus,
|
||||
nextPollCommand: `hwlab-cli client agent result ${traceId} --base-url ${baseUrl}`,
|
||||
nextPollCommand: traceLookup.commands.result,
|
||||
traceCommand: traceLookup.commands.trace,
|
||||
inspectCommand: traceLookup.commands.inspect,
|
||||
sessionCommand: traceLookup.commands.sessionStatus,
|
||||
traceLookup,
|
||||
sessionResponse: compactObject(session.body),
|
||||
error: result.error
|
||||
});
|
||||
@@ -739,19 +762,13 @@ async function collectAgentTraceEvidence(context: CaseContext, run: PreparedCase
|
||||
const agent = run.agent;
|
||||
const traceId = text(agent?.traceId);
|
||||
if (!agent || !traceId) {
|
||||
const trace: AgentTraceStage = clean({ traceId, status: "skipped_no_trace_id", httpStatus: 0, commandCount: 0, hwpodCommandCount: 0, hwpodBuildCommandCount: 0, keilJobCandidates: [], commands: [] });
|
||||
const trace: AgentTraceStage = clean({ traceId, status: "skipped_no_trace_id", httpStatus: 0, source: "caserun-identity", lookupOnly: true, commandCount: 0, hwpodCommandCount: 0, hwpodBuildCommandCount: 0, keilJobCandidates: [], commands: [] });
|
||||
const nextRun = await updateRun(context, run, { agentTrace: trace, stage: "agent-trace-collected" });
|
||||
return { run: nextRun, trace };
|
||||
}
|
||||
const traceUrl = `${agent.baseUrl}/v1/agent/chat/trace/${encodeURIComponent(traceId)}`;
|
||||
const response = await caseFetchJson(context, traceUrl, { method: "GET", headers: caseAgentHeaders(context) }, "agent_trace_fetch");
|
||||
if (!response.ok) {
|
||||
const trace: AgentTraceStage = clean({ traceId, status: "fetch_failed", httpStatus: response.status, commandCount: 0, hwpodCommandCount: 0, hwpodBuildCommandCount: 0, keilJobCandidates: [], commands: [], error: response.error });
|
||||
const nextRun = await updateRun(context, run, { agentTrace: trace, stage: "agent-trace-collected" });
|
||||
return { run: nextRun, trace };
|
||||
}
|
||||
await writeJson(path.join(run.runDir, "agent-trace.json"), response.body);
|
||||
const trace = summarizeAgentTrace(traceId, response.status, response.body);
|
||||
const traceBody = agentTraceIdentityArtifact(run, agent);
|
||||
await writeJson(path.join(run.runDir, "agent-trace.json"), traceBody);
|
||||
const trace = summarizeAgentTrace(traceId, 0, traceBody);
|
||||
const nextRun = await updateRun(context, run, { agentTrace: trace, stage: "agent-trace-collected" });
|
||||
return { run: nextRun, trace };
|
||||
}
|
||||
@@ -823,6 +840,56 @@ async function submitAgentTask(context: CaseContext, input: { baseUrl: string; p
|
||||
}, "agent_task_submit");
|
||||
}
|
||||
|
||||
function agentTraceLookup(input: { baseUrl: string; traceId: string; sessionId?: string; conversationId?: string; threadId?: string }) {
|
||||
const traceId = input.traceId;
|
||||
const base = input.baseUrl;
|
||||
const commands = clean({
|
||||
result: `hwlab-cli client agent result ${traceId} --base-url ${base}`,
|
||||
trace: `hwlab-cli client agent trace ${traceId} --base-url ${base} --render web`,
|
||||
traceFull: `hwlab-cli client agent trace ${traceId} --base-url ${base} --render web --full`,
|
||||
inspect: `hwlab-cli client agent inspect --trace-id ${traceId} --base-url ${base}`,
|
||||
sessionStatus: input.sessionId ? `hwlab-cli client agent session status ${input.sessionId} --base-url ${base}` : undefined
|
||||
});
|
||||
return clean({
|
||||
source: "hwlab-cli.client.agent",
|
||||
strategy: "id_plus_existing_cli",
|
||||
note: "CaseRun records trace/session identity and reuses existing HWLAB Web-equivalent agent CLI for trace/result/inspect; it does not implement a separate trace query path.",
|
||||
baseUrl: base,
|
||||
traceId,
|
||||
sessionId: input.sessionId,
|
||||
conversationId: input.conversationId,
|
||||
threadId: input.threadId,
|
||||
commands
|
||||
});
|
||||
}
|
||||
|
||||
function agentTraceIdentityArtifact(run: PreparedCaseRun, agent: AgentRunStage) {
|
||||
const traceId = text(agent.traceId);
|
||||
const lookup = agent.traceLookup ?? agentTraceLookup({ baseUrl: agent.baseUrl, traceId, sessionId: agent.sessionId, conversationId: agent.conversationId, threadId: agent.threadId });
|
||||
const resultBody = agent.result?.body && typeof agent.result.body === "object" ? agent.result.body : agent.result;
|
||||
return clean({
|
||||
contractVersion: "hwpod-case-run-agent-trace-identity-v1",
|
||||
source: "caserun-identity",
|
||||
lookupOnly: true,
|
||||
status: text(resultBody?.status ?? agent.stageStatus) || "recorded",
|
||||
traceId,
|
||||
sessionId: agent.sessionId,
|
||||
conversationId: agent.conversationId,
|
||||
threadId: agent.threadId,
|
||||
providerProfile: agent.providerProfile,
|
||||
projectId: agent.projectId,
|
||||
resultStatus: text(resultBody?.status),
|
||||
finalResponse: resultBody?.finalResponse ?? resultBody?.reply ?? null,
|
||||
traceSummary: resultBody?.traceSummary ?? null,
|
||||
terminalEvidence: resultBody?.terminalEvidence ?? null,
|
||||
agentRun: compactObject(resultBody?.agentRun ?? resultBody?.traceSummary?.agentRun ?? null),
|
||||
lookup,
|
||||
hint: "Use lookup.commands.trace or lookup.commands.result to query the full Web-equivalent AgentRun trace with the existing HWLAB CLI.",
|
||||
runId: run.runId,
|
||||
caseId: run.caseId
|
||||
});
|
||||
}
|
||||
|
||||
async function agentWorkspaceFilesForRun(run: PreparedCaseRun): Promise<AgentWorkspaceFile[]> {
|
||||
const content = await readFile(run.specPath, "utf8");
|
||||
return [{ path: ".hwlab/hwpod-spec.yaml", content, encoding: "utf8" }];
|
||||
@@ -980,6 +1047,9 @@ function summarizeAgentTrace(traceId: string, httpStatus: number, body: any): Ag
|
||||
traceId,
|
||||
status: text(traceBody?.status ?? body?.status) || "recorded",
|
||||
httpStatus,
|
||||
source: text(traceBody?.source ?? body?.source),
|
||||
lookupOnly: traceBody?.lookupOnly === true || body?.lookupOnly === true ? true : undefined,
|
||||
lookup: compactObject(traceBody?.lookup ?? body?.lookup ?? null),
|
||||
sourceEventCount: numberOption(traceBody?.sourceEventCount ?? traceBody?.traceSummary?.sourceEventCount ?? body?.sourceEventCount),
|
||||
renderedRowCount: rows.length || numberOption(traceBody?.renderedRowCount),
|
||||
commandCount: commands.length,
|
||||
@@ -987,7 +1057,7 @@ function summarizeAgentTrace(traceId: string, httpStatus: number, body: any): Ag
|
||||
hwpodBuildCommandCount,
|
||||
keilJobCandidates: uniqueStrings(Array.from(searchable.matchAll(/20\d{6}_\d{6}_[0-9a-f]{8}/giu)).map((match) => match[0])),
|
||||
terminalStatus: text(traceBody?.terminalStatus ?? traceBody?.traceSummary?.terminalStatus ?? body?.terminalStatus),
|
||||
agentRun: compactObject(traceBody?.agentRun ?? traceBody?.traceSummary?.agentRun ?? body?.agentRun ?? null),
|
||||
agentRun: compactObject(traceBody?.agentRun ?? traceBody?.traceSummary?.agentRun ?? body?.agentRun ?? traceBody?.terminalEvidence?.agentRun ?? null),
|
||||
commands
|
||||
});
|
||||
}
|
||||
@@ -1164,7 +1234,7 @@ function agentTaskSummary(agentTask: PreparedAgentTask) {
|
||||
}
|
||||
|
||||
function agentSummary(agent: AgentRunStage) {
|
||||
return clean({ stageStatus: agent.stageStatus, baseUrl: agent.baseUrl, projectId: agent.projectId, providerProfile: agent.providerProfile, conversationId: agent.conversationId, sessionId: agent.sessionId, threadId: agent.threadId, traceId: agent.traceId, promptPath: agent.promptPath, promptSha256: agent.promptSha256, polls: agent.polls, timedOut: agent.timedOut, timeoutMs: agent.timeoutMs, pollIntervalMs: agent.pollIntervalMs, lastHttpStatus: agent.lastHttpStatus, error: agent.error });
|
||||
return clean({ stageStatus: agent.stageStatus, baseUrl: agent.baseUrl, projectId: agent.projectId, providerProfile: agent.providerProfile, conversationId: agent.conversationId, sessionId: agent.sessionId, threadId: agent.threadId, traceId: agent.traceId, promptPath: agent.promptPath, promptSha256: agent.promptSha256, polls: agent.polls, timedOut: agent.timedOut, timeoutMs: agent.timeoutMs, pollIntervalMs: agent.pollIntervalMs, lastHttpStatus: agent.lastHttpStatus, nextPollCommand: agent.nextPollCommand, traceCommand: agent.traceCommand, inspectCommand: agent.inspectCommand, sessionCommand: agent.sessionCommand, traceLookup: agent.traceLookup, error: agent.error });
|
||||
}
|
||||
|
||||
function traceSummary(trace: AgentTraceStage) {
|
||||
@@ -1172,6 +1242,9 @@ function traceSummary(trace: AgentTraceStage) {
|
||||
traceId: trace.traceId,
|
||||
status: trace.status,
|
||||
httpStatus: trace.httpStatus,
|
||||
source: trace.source,
|
||||
lookupOnly: trace.lookupOnly,
|
||||
lookup: trace.lookup,
|
||||
sourceEventCount: trace.sourceEventCount,
|
||||
renderedRowCount: trace.renderedRowCount,
|
||||
commandCount: trace.commandCount,
|
||||
@@ -1418,11 +1491,27 @@ function prepareSummary(run: PreparedCaseRun) {
|
||||
}
|
||||
|
||||
function buildSummary(evidence: any) {
|
||||
return { status: evidence.status, autoEvaluation: false, agentTraceCommandCount: evidence.agentTrace?.commandCount ?? 0, agentTraceHwpodCommandCount: evidence.agentTrace?.hwpodCommandCount ?? 0, agentStage: agentStageSummary(evidence), hwpodExitCode: evidence.hwpod?.exitCode ?? null, hwpodSource: evidence.hwpod?.source ?? null, jobId: evidence.keilJob?.jobId ?? null, keilStatus: evidence.keilJob?.status ?? null, artifacts: evidence.artifacts ?? [] };
|
||||
return { status: evidence.status, autoEvaluation: false, traceLookup: evidence.traceLookup ?? null, agentTraceCommandCount: evidence.agentTrace?.commandCount ?? 0, agentTraceHwpodCommandCount: evidence.agentTrace?.hwpodCommandCount ?? 0, agentStage: agentStageSummary(evidence), hwpodExitCode: evidence.hwpod?.exitCode ?? null, hwpodSource: evidence.hwpod?.source ?? null, jobId: evidence.keilJob?.jobId ?? null, keilStatus: evidence.keilJob?.status ?? null, artifacts: evidence.artifacts ?? [] };
|
||||
}
|
||||
|
||||
function agentTraceLookupForRun(run: PreparedCaseRun, evidence?: any) {
|
||||
const agent = run.agent ?? evidence?.agent;
|
||||
const traceId = text(agent?.traceId ?? evidence?.agentTrace?.traceId ?? run.agentTrace?.traceId);
|
||||
if (!traceId) return null;
|
||||
const baseUrl = text(agent?.baseUrl ?? run.agent?.baseUrl ?? evidence?.agent?.baseUrl) || runtimeWebUrl(run.definition) || DEFAULT_WEB_URL;
|
||||
return agentTraceLookup({ baseUrl, traceId, sessionId: text(agent?.sessionId), conversationId: text(agent?.conversationId), threadId: text(agent?.threadId) });
|
||||
}
|
||||
|
||||
function runtimeWebUrl(definition: Record<string, unknown>) {
|
||||
const runtime = definition.runtime;
|
||||
if (!runtime || typeof runtime !== "object" || Array.isArray(runtime)) return "";
|
||||
return text((runtime as Record<string, unknown>).webUrl);
|
||||
}
|
||||
|
||||
function renderEvidenceSummary(evidence: any) {
|
||||
const agentStage = agentStageSummary(evidence);
|
||||
const traceLookup = evidence.traceLookup ?? evidence.agent?.traceLookup;
|
||||
const traceCommands = traceLookup?.commands ?? {};
|
||||
const stageTable = agentStage.commands.length > 0 ? [
|
||||
``,
|
||||
`## Agent HWPOD Raw Steps`,
|
||||
@@ -1431,7 +1520,7 @@ function renderEvidenceSummary(evidence: any) {
|
||||
`|---|---:|---:|---|---|`,
|
||||
...agentStage.commands.map((command: any) => `| ${command.kind} | ${command.status || ""} | ${command.exitCode ?? ""} | \`${markdownTableText(command.command, 160)}\` | ${markdownTableText(command.detail, 180)} |`)
|
||||
].join("\n") : "";
|
||||
return `# HWPOD CaseRun ${evidence.caseId}\n\n- runId: ${evidence.runId}\n- status: ${evidence.status}\n- autoEvaluation: false\n- compileOnly: ${evidence.compileOnly}\n- subjectRepoLocalPath: ${evidence.subject?.repoLocalPath ?? ""}\n- subjectCommitId: ${evidence.subject?.commitId ?? ""}\n- subjectWorktreePath: ${evidence.subject?.worktreePath ?? ""}\n- agentTraceId: ${evidence.agent?.traceId ?? ""}\n- agentSessionId: ${evidence.agent?.sessionId ?? ""}\n- agentTraceCommandCount: ${evidence.agentTrace?.commandCount ?? 0}\n- agentTraceHwpodCommandCount: ${evidence.agentTrace?.hwpodCommandCount ?? 0}\n- agentTraceHwpodBuildCommandCount: ${evidence.agentTrace?.hwpodBuildCommandCount ?? 0}\n- agentStageCommandCount: ${agentStage.commands.length}\n- agentStageKinds: ${agentStage.kinds.join(", ")}\n- diffPatchPath: ${evidence.agentDiff?.diffPatchPath ?? ""}\n- runnerHwpodSource: ${evidence.hwpod?.source ?? ""}\n- hwpodExitCode: ${evidence.hwpod?.exitCode ?? ""}\n- jobId: ${evidence.keilJob?.jobId ?? ""}\n- runnerPostAgentCompileCheck: recorded\n${stageTable}\n`;
|
||||
return `# HWPOD CaseRun ${evidence.caseId}\n\n- runId: ${evidence.runId}\n- status: ${evidence.status}\n- autoEvaluation: false\n- compileOnly: ${evidence.compileOnly}\n- subjectRepoLocalPath: ${evidence.subject?.repoLocalPath ?? ""}\n- subjectCommitId: ${evidence.subject?.commitId ?? ""}\n- subjectWorktreePath: ${evidence.subject?.worktreePath ?? ""}\n- agentTraceId: ${evidence.agent?.traceId ?? ""}\n- agentSessionId: ${evidence.agent?.sessionId ?? ""}\n- traceLookupStrategy: ${traceLookup?.strategy ?? ""}\n- traceCommand: ${traceCommands.trace ?? ""}\n- resultCommand: ${traceCommands.result ?? ""}\n- inspectCommand: ${traceCommands.inspect ?? ""}\n- agentTraceCommandCount: ${evidence.agentTrace?.commandCount ?? 0}\n- agentTraceHwpodCommandCount: ${evidence.agentTrace?.hwpodCommandCount ?? 0}\n- agentTraceHwpodBuildCommandCount: ${evidence.agentTrace?.hwpodBuildCommandCount ?? 0}\n- agentStageCommandCount: ${agentStage.commands.length}\n- agentStageKinds: ${agentStage.kinds.join(", ")}\n- diffPatchPath: ${evidence.agentDiff?.diffPatchPath ?? ""}\n- runnerHwpodSource: ${evidence.hwpod?.source ?? ""}\n- hwpodExitCode: ${evidence.hwpod?.exitCode ?? ""}\n- jobId: ${evidence.keilJob?.jobId ?? ""}\n- runnerPostAgentCompileCheck: recorded\n${stageTable}\n`;
|
||||
}
|
||||
|
||||
function agentStageSummary(evidence: any) {
|
||||
@@ -1513,6 +1602,7 @@ function collectSummaryFromArchive(run: PreparedCaseRun, evidence: any, archive:
|
||||
artifactCount: archive.files.length + 1,
|
||||
skippedFiles: archive.skippedFiles,
|
||||
trace: archive.manifest.trace ?? null,
|
||||
traceLookup: archive.manifest.traceLookup ?? agentTraceLookupForRun(run, evidence),
|
||||
agentStage: archive.manifest.agentStage ?? agentStageSummary(evidence),
|
||||
status: evidence.status,
|
||||
autoEvaluation: false
|
||||
@@ -1528,10 +1618,14 @@ async function writeReadableAgentArtifacts(caseRepoRunDir: string, run: Prepared
|
||||
const events = arrayOfObjects(traceBody?.events ?? traceBody?.trace?.events ?? traceBody?.sourceEvents);
|
||||
const rows = traceDisplayRows(traceBody ?? {}, events);
|
||||
const finalResponse = agentFinalResponse(run, evidence, traceBody, rows);
|
||||
const traceLookup = traceBody?.lookup ?? evidence.traceLookup ?? agentTraceLookupForRun(run, evidence);
|
||||
const diffPatch = await readTextIfExists(evidence.agentDiff?.diffPatchPath ?? run.agentDiff?.diffPatchPath ?? path.join(run.runDir, "agent-diff.patch"));
|
||||
const messages = clean({
|
||||
contractVersion: "hwpod-case-run-agent-messages-v1",
|
||||
renderer: CASE_TRACE_RENDERER,
|
||||
traceSource: text(traceBody?.source),
|
||||
lookupOnly: traceBody?.lookupOnly === true ? true : undefined,
|
||||
traceLookup,
|
||||
traceId: evidence.agent?.traceId ?? run.agent?.traceId ?? traceBody?.traceId ?? run.agentTrace?.traceId,
|
||||
conversationId: evidence.agent?.conversationId ?? run.agent?.conversationId,
|
||||
sessionId: evidence.agent?.sessionId ?? run.agent?.sessionId,
|
||||
@@ -1615,6 +1709,11 @@ function renderAgentTranscript(run: PreparedCaseRun, evidence: any, messages: an
|
||||
`- sessionId: ${messages.sessionId ?? ""}`,
|
||||
`- threadId: ${messages.threadId ?? ""}`,
|
||||
`- renderer: ${CASE_TRACE_RENDERER}`,
|
||||
`- traceLookupStrategy: ${messages.traceLookup?.strategy ?? ""}`,
|
||||
`- traceCommand: ${messages.traceLookup?.commands?.trace ?? ""}`,
|
||||
`- resultCommand: ${messages.traceLookup?.commands?.result ?? ""}`,
|
||||
`- inspectCommand: ${messages.traceLookup?.commands?.inspect ?? ""}`,
|
||||
`- lookupOnly: ${messages.lookupOnly === true}`,
|
||||
`- finalResponse: ${messages.finalResponse?.present === true ? "present" : "null"}`,
|
||||
`- autoEvaluation: false`,
|
||||
``,
|
||||
@@ -1726,6 +1825,7 @@ function caseRunArtifactManifest(context: CaseContext, run: PreparedCaseRun, evi
|
||||
sourceRunDir: run.runDir,
|
||||
caseRepoRunDir: path.join(run.caseRepo, caseRegistryRunRel(run)),
|
||||
trace: caseRunArtifactTrace(run, evidence),
|
||||
traceLookup: evidence.traceLookup ?? agentTraceLookupForRun(run, evidence),
|
||||
subject: evidence.subject ?? run.subject,
|
||||
agent: clean({
|
||||
providerProfile: evidence.agent?.providerProfile ?? run.agent?.providerProfile,
|
||||
@@ -1767,6 +1867,9 @@ function caseRunArtifactTrace(run: PreparedCaseRun, evidence: any) {
|
||||
threadId: evidence.agent?.threadId ?? run.agent?.threadId,
|
||||
providerProfile: evidence.agent?.providerProfile ?? run.agent?.providerProfile,
|
||||
projectId: evidence.agent?.projectId ?? run.agent?.projectId,
|
||||
source: evidence.agentTrace?.source ?? run.agentTrace?.source,
|
||||
lookupOnly: evidence.agentTrace?.lookupOnly ?? run.agentTrace?.lookupOnly,
|
||||
lookup: evidence.traceLookup ?? evidence.agentTrace?.lookup ?? run.agentTrace?.lookup ?? agentTraceLookupForRun(run, evidence),
|
||||
status: evidence.agentTrace?.status ?? run.agentTrace?.status,
|
||||
terminalStatus: evidence.agentTrace?.terminalStatus ?? run.agentTrace?.terminalStatus,
|
||||
commandCount: evidence.agentTrace?.commandCount ?? run.agentTrace?.commandCount,
|
||||
|
||||
Reference in New Issue
Block a user