fix: 修复纯 Kafka Trace 与原始事件观察窗

This commit is contained in:
root
2026-07-10 15:42:54 +02:00
parent 8368c5801c
commit 3a4912906b
30 changed files with 976 additions and 97 deletions
+12 -1
View File
@@ -587,7 +587,8 @@
"workbenchUiPolicy": {
"type": "object",
"properties": {
"traceTimeline": { "$ref": "#/$defs/workbenchTraceTimelinePolicy" }
"traceTimeline": { "$ref": "#/$defs/workbenchTraceTimelinePolicy" },
"rawHwlabEventWindow": { "$ref": "#/$defs/workbenchRawHwlabEventWindowPolicy" }
},
"additionalProperties": false
},
@@ -599,6 +600,16 @@
},
"additionalProperties": false
},
"workbenchRawHwlabEventWindowPolicy": {
"type": "object",
"required": ["enabled", "maxEntries", "maxRetainedBytes"],
"properties": {
"enabled": { "type": "boolean" },
"maxEntries": { "type": "integer", "minimum": 1 },
"maxRetainedBytes": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
"externalPostgres": {
"type": "object",
"required": ["enabled", "serviceName", "endpointAddress", "port"],
+4
View File
@@ -374,6 +374,10 @@ lanes:
traceTimeline:
autoExpandRunning: false
autoCollapseTerminal: false
rawHwlabEventWindow:
enabled: true
maxEntries: 200
maxRetainedBytes: 1048576
observability:
traceExplorerUrlTemplate: /v1/workbench/traces/{trace_id}/events
prometheusOperatorResources: false
@@ -0,0 +1,47 @@
# R2.1 纯 Kafka Trace 序列权威修复报告
## 上下文
- 前端根因:
- [HWLAB #2475](https://github.com/pikasTech/HWLAB/issues/2475#issuecomment-4935381417)
- 运行面证据:
- [HWLAB #2477](https://github.com/pikasTech/HWLAB/issues/2477#issuecomment-4935385934)
## 完成结果
- 共享 Trace renderer 按 `runnerTrace.eventSource` 选择序列权威:
- `hwlab-kafka-sse` 使用 `sourceSeq`
- `trace-api` 和其他 durable 路径继续只使用 `projectedSeq`
- live Kafka 事件保持前端 ingress 数组顺序:
- 不按 `sourceSeq` 重新排序;
- 优先以 `sourceEventId` 去重;
- 缺少该字段时使用 `runId + sourceSeq`
- 不伪造 `projectedSeq`
- Trace 行显式携带 `sequenceAuthority`
- DOM 通用属性为 `data-event-seq``data-event-seq-authority`
- 只有 durable 行继续暴露 `data-projected-seq`
- live Trace 合并保持:
- source identity 去重;
- terminal 事件不清空既有事件;
- `eventCount` 等于已保存事件数。
## 验证
- production mapper 验证:
- inner event 携带 `sourceSeq`
- inner event 不产生 `projectedSeq`
- renderer 与状态验证:
- source-only assistant、tool 和 terminal 能生成可读行;
- 行顺序与 ingress 顺序一致;
- durable Trace 缺少 `projectedSeq` 时仍拒绝渲染;
- 重复 `sourceEventId` 不产生重复行。
- 定向测试结果:
- Bun 后端、renderer、merge 和 SSE API`50 pass`
- TraceTimeline 与调试面板组件:`4 pass`
- v03 GitOps render 目标用例:`1 pass`
## 边界
- 未启用或调用 projector、read model、snapshot、sync、recovery replay、gap-fill 或 polling。
- 未部署运行面。
- 未修改父 issue [HWLAB #2474](https://github.com/pikasTech/HWLAB/issues/2474)。
@@ -0,0 +1,76 @@
# R2.2 原始 HWLAB Event 观察窗报告
## 上下文
- 主问题:
- [HWLAB #2474](https://github.com/pikasTech/HWLAB/issues/2474)
- 前端设计与最小 seam
- [HWLAB #2475](https://github.com/pikasTech/HWLAB/issues/2475#issuecomment-4935381417)
## 完成结果
- 现有产品 EventSource 增加单一 ingress tee
- 每个命名 `hwlab.event.v1` frame 在 decode 和 coalesce 前记录一次;
- 同一次 decode 结果继续进入原生产 reducer;
- 未增加产品 EventSource、Kafka consumer 或第二事实源。
- 原始观察状态为 bounded 内存:
- 按到达顺序保留完整 `MessageEvent.data`
- 不按 trace、event type 或 reducer 结果做客户端过滤;
- invalid JSON 和非对象 JSON 保留原文并显示拒绝原因;
- 淘汰完整 envelope,不截断仍被保留的 envelope;
- 不写 localStorage、数据库或 canonical Workbench store。
- 隔离调试卡片增加独立子标签:
- `Trace 卡片` 保留隔离 debug replay 和生产 reducer 卡片;
- `原始 HWLAB Event` 使用只读纯文本框显示同一产品 ingress;
- 切换标签不创建或关闭网络连接;
- 清空观察窗只清原始内存 buffer。
- R2.3 v2 UI 集成点已预留:
- 支持 `replayId` 和 output barrier query 参数;
- 接收 `replay-complete.result`
- 展示 server `scanned/matched/delivered`
- 用明确标注的 client local overlay 展示 `received/decoded/applied`
## YAML-first
- owning YAML
- `deploy/deploy.yaml#lanes.v03.workbench.rawHwlabEventWindow`
- 独立字段:
- `enabled`
- `maxEntries`
- `maxRetainedBytes`
- 渲染链:
- deploy schema 校验字段形状;
- GitOps renderer 生成 Cloud Web env
- Cloud Web runtime 注入浏览器配置;
- 前端只读取显式值,不设置容量默认值。
- 特例分类:
- `keep-domain-special`:观察窗保持管理员可见、内存只读;
- `remove-code-default`:容量与开关没有代码默认值;
- `legacy-retire`:本任务没有新增或保留旧入口。
## 验证
- ingress 与 buffer
- 每个产品 frame 恰好产生一条 raw ingress 和一次 reducer delivery
- invalid JSON 可见但不进入 reducer
- entry/byte eviction 均保持完整 envelope。
- UI
- 两个子标签、局部清空和 raw 文本顺序通过;
- 标签切换期间 EventSource 实例数保持不变。
- 配置与 runtime
- Cloud Web runtime`18 pass`
- v03 GitOps render 目标用例:`1 pass`
- deploy 合同:`15` 个 JSON 文件验证通过。
- 定向功能测试:
- Bun 后端、renderer、merge 和 SSE API`50 pass`
- TraceTimeline 与调试面板组件:`4 pass`
- 已知基线失败:
- renderer 全套仍有两个与本变更无关的旧断言失败;
- Web check 的 source-shape、显式 any 扫描和 Vite build 通过;
- `workbench-isolated-kafka-debug.test.ts:61` 仍有 R2.3 分支已有的类型断言错误。
## 边界
- 未部署 NC01/v03。
- 未修改父 issue [HWLAB #2474](https://github.com/pikasTech/HWLAB/issues/2474)。
- 产品链继续保持纯 Kafka 实时,不增加补洞或投影路径。
@@ -19,10 +19,10 @@
## R2 [in_progress]
修复纯 Kafka 实时 Trace 可读性、隔离调试分层诊断与同一 SSE ingress 原始 HWLAB Event 观察窗;保持产品链无 projector、read model、snapshot、sync、recovery replay 或 polling。上下文:[HWLAB #2474](https://github.com/pikasTech/HWLAB/issues/2474) 与 [调查结论](https://github.com/pikasTech/HWLAB/issues/2474#issuecomment-4935408473),完成任务后将详细报告写入[任务报告](./details/pure-kafka-live-single-step-debug/R2_Task_Report.md)。
### R2.1 [in_progress]
### R2.1 [completed]
按 eventSource 修复 Trace 序列权威:live Kafka 保留 ingress 顺序并以 sourceEventId 或 runId+sourceSeq 识别事件,不伪造 projectedSeq;补 sourceSeq-only running/assistant/tool/terminal、去重与 terminal 保留合同。上下文:[前端根因 #2475](https://github.com/pikasTech/HWLAB/issues/2475#issuecomment-4935381417)、[运行面证据 #2477](https://github.com/pikasTech/HWLAB/issues/2477#issuecomment-4935385934)。依赖:无。验证:真实 mapper envelope 的 eventCount 等于 stored events、readableRows 大于 0projection 保持关闭,完成任务后将详细报告写入[任务报告](./details/pure-kafka-live-single-step-debug/R2.1_Task_Report.md)。
### R2.2
### R2.2 [completed]
在同一产品 EventSource 的 decode/coalesce 前 tee 原始 frame,并在隔离调试卡片增加 Trace 卡片/原始 HWLAB Event 子标签;raw 按到达顺序写入 bounded 内存、暴露 invalid JSONYAML-first 开关和容量独立配置,不新增 SSE 或 Kafka consumer。上下文:[主 issue #2474](https://github.com/pikasTech/HWLAB/issues/2474)、[前端设计 #2475](https://github.com/pikasTech/HWLAB/issues/2475#issuecomment-4935381417)。依赖:无,可与 Trace 序列权威修复并行。验证:每个产品 frame 恰好 raw+1 且 reducer 一次,切换标签不新增 EventSourceeviction/invalid-json/runtime contract 通过,完成任务后将详细报告写入[任务报告](./details/pure-kafka-live-single-step-debug/R2.2_Task_Report.md)。
### R2.3
@@ -88,6 +88,8 @@ test("projects AgentRun assistant_message Kafka event into HWLAB trace event", (
assert.equal(projected.context.sourceSeq, 7);
assert.equal(projected.context.agentRunEventType, "assistant_message");
assert.equal(projected.event.type, "assistant");
assert.equal(projected.event.sourceSeq, 7);
assert.equal(Object.prototype.hasOwnProperty.call(projected.event, "projectedSeq"), false);
assert.equal(projected.event.label, "agentrun:assistant:message");
assert.equal(projected.event.status, "running");
assert.equal(projected.event.text, "partial answer");
+12 -1
View File
@@ -396,7 +396,12 @@ function workbenchRuntimeConfigFromEnv() {
projectionRealtime: requiredWorkbenchRealtimeFeature(process.env.HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED, "HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED")
},
debugCapabilities: {
isolatedKafka: requiredWorkbenchRealtimeFeature(process.env.HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED, "HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED")
isolatedKafka: requiredWorkbenchRealtimeFeature(process.env.HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED, "HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED"),
rawHwlabEventWindow: {
enabled: requiredWorkbenchRealtimeFeature(process.env.HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED, "HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED"),
maxEntries: requiredPositiveWorkbenchInteger(process.env.HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES, "HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES"),
maxRetainedBytes: requiredPositiveWorkbenchInteger(process.env.HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES, "HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES")
}
}
};
const autoExpandRunning = parseEnvBoolean(process.env.HWLAB_WORKBENCH_TRACE_AUTO_EXPAND_RUNNING);
@@ -416,6 +421,12 @@ function requiredWorkbenchRealtimeFeature(value, name) {
throw new Error(`${name} is required and must be explicitly true or false`);
}
function requiredPositiveWorkbenchInteger(value, name) {
const number = Number(value);
if (!Number.isInteger(number) || number <= 0) throw new Error(`${name} is required and must be a positive integer`);
return number;
}
function traceExplorerUrlTemplateFromEnv(value) {
if (value === undefined || value === null || value === "") return null;
const template = String(value).trim();
@@ -432,7 +432,10 @@ test("cloud web serves client deep links through the Vue shell", async () => {
HWLAB_CLOUD_WEB_DISPLAY_TIME_LABEL: "北京时间",
HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED: "true",
HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED: "false",
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true"
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES: "200",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES: "1048576"
});
const root = await mkdtemp(path.join(os.tmpdir(), "hwlab-cloud-web-runtime-"));
await writeFile(path.join(root, "index.html"), "<div id=\"root\"></div>\n", "utf8");
@@ -463,7 +466,7 @@ test("cloud web serves client deep links through the Vue shell", async () => {
const html = await response.text();
assert.match(html, /<div id="root"><\/div>/u, route);
assert.match(html, /HWLAB_CLOUD_WEB_CONFIG/u, route);
assert.match(html, /"debugCapabilities":\{"isolatedKafka":true\}/u, route);
assert.match(html, /"debugCapabilities":\{"isolatedKafka":true,"rawHwlabEventWindow":\{"enabled":true,"maxEntries":200,"maxRetainedBytes":1048576\}\}/u, route);
}
const assetResponse = await fetch(`${serverUrl(cloudWeb)}/asset.txt`, {
@@ -491,7 +494,10 @@ test("cloud web injects trace explorer runtime config from env", async () => {
HWLAB_WORKBENCH_TRACE_EXPLORER_URL_TEMPLATE: "/v1/workbench/traces/{trace_id}/events",
HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED: "true",
HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED: "false",
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true"
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES: "200",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES: "1048576"
});
const root = await mkdtemp(path.join(os.tmpdir(), "hwlab-cloud-web-runtime-"));
await writeFile(path.join(root, "index.html"), "<html><head></head><body><div id=\"root\"></div></body></html>\n", "utf8");
@@ -598,7 +604,10 @@ test("cloud web OpenCode frame-url endpoint mints traced tickets", async () => {
HWLAB_CLOUD_WEB_OPENCODE_URL: "https://opencode.example.test",
HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED: "true",
HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED: "false",
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true"
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES: "200",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES: "1048576"
});
const cloudApiRequests = [];
const opencodeRequests = [];
@@ -1041,7 +1050,10 @@ test("cloud web OpenCode proxy accepts short-lived tickets minted by the shell",
HWLAB_CLOUD_WEB_OPENCODE_URL: "https://opencode.example.test",
HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED: "true",
HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED: "false",
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true"
HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED: "true",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES: "200",
HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES: "1048576"
});
const root = await mkdtemp(path.join(os.tmpdir(), "hwlab-cloud-web-runtime-"));
await writeFile(path.join(root, "index.html"), "<html><head></head><body><div id=\"root\"></div></body></html>\n", "utf8");
+3
View File
@@ -660,6 +660,9 @@ test("v03 render keeps node identity as data instead of generated structure", as
assert.equal(cloudWebEnv.get("HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED"), "true");
assert.equal(cloudWebEnv.get("HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED"), "false");
assert.equal(cloudWebEnv.get("HWLAB_WORKBENCH_KAFKA_DEBUG_REPLAY_ENABLED"), "true");
assert.equal(cloudWebEnv.get("HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED"), "1");
assert.equal(cloudWebEnv.get("HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES"), "200");
assert.equal(cloudWebEnv.get("HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES"), "1048576");
assert.match(cloudApiEnv.get("HWLAB_ENVIRONMENT_IMAGE") ?? "", /hwlab-cloud-api-env:env-stale/u);
assert.deepEqual(cloudApiEnvEntries.get("HWLAB_SECRET_PLANE_SMOKE")?.valueFrom?.secretKeyRef, { name: "hwlab-secret-plane-smoke", key: "password", optional: true });
assert.ok((cloudApi?.spec?.template?.spec?.volumes ?? []).some((volume) => volume.name === "hwpod-preinstalled-specs" && volume.configMap?.name === "hwlab-v03-hwpod-preinstalled-specs"));
+17
View File
@@ -1136,6 +1136,17 @@ function runtimeWorkbenchTraceTimelinePolicy(deploy, profile) {
return Object.keys(result).length > 0 ? result : null;
}
function runtimeWorkbenchRawHwlabEventWindowPolicy(deploy, profile) {
if (!isRuntimeLane(profile)) return null;
const policy = runtimeLaneConfig(deploy, profile)?.workbench?.rawHwlabEventWindow;
if (!policy || typeof policy !== "object" || Array.isArray(policy)) return null;
return {
enabled: policy.enabled,
maxEntries: policy.maxEntries,
maxRetainedBytes: policy.maxRetainedBytes
};
}
function runtimeTraceExplorerUrlTemplate(deploy, profile) {
if (!isRuntimeLane(profile)) return null;
const template = runtimeLaneConfig(deploy, profile)?.observability?.traceExplorerUrlTemplate;
@@ -1586,6 +1597,12 @@ function transformWorkloads({ workloads, deploy, catalog, source, sourceBranch =
if (typeof traceTimelinePolicy?.autoCollapseTerminal === "boolean") {
upsertEnv(container.env, "HWLAB_WORKBENCH_TRACE_AUTO_COLLAPSE_TERMINAL", booleanEnv(traceTimelinePolicy.autoCollapseTerminal));
}
const rawHwlabEventWindowPolicy = runtimeWorkbenchRawHwlabEventWindowPolicy(deploy, profile);
if (rawHwlabEventWindowPolicy) {
upsertEnv(container.env, "HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_ENABLED", booleanEnv(rawHwlabEventWindowPolicy.enabled));
upsertEnv(container.env, "HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_ENTRIES", String(rawHwlabEventWindowPolicy.maxEntries));
upsertEnv(container.env, "HWLAB_WORKBENCH_RAW_HWLAB_EVENT_WINDOW_MAX_RETAINED_BYTES", String(rawHwlabEventWindowPolicy.maxRetainedBytes));
}
const traceExplorerUrlTemplate = runtimeTraceExplorerUrlTemplate(deploy, profile);
if (traceExplorerUrlTemplate) {
upsertEnv(container.env, "HWLAB_WORKBENCH_TRACE_EXPLORER_URL_TEMPLATE", traceExplorerUrlTemplate);
+74 -38
View File
@@ -1,6 +1,7 @@
export interface TraceEventRow {
rowId: string;
seq: number | null;
sequenceAuthority: TraceSequenceAuthority;
tone: "ok" | "blocked" | "warn" | "source";
header: string;
body: string | null;
@@ -11,13 +12,25 @@ export interface TraceEventRow {
type TraceEvent = Record<string, unknown>;
type TraceClockFormatter = (value: string) => string;
export type TraceSequenceAuthority = "projected" | "source";
export interface TraceDisplayRowsOptions {
formatClock?: TraceClockFormatter;
}
interface ResolvedTraceDisplayRowsOptions extends TraceDisplayRowsOptions {
sequenceAuthority: TraceSequenceAuthority;
}
export function traceDisplayRows(trace: Record<string, unknown> = {}, events: TraceEvent[] = [], options: TraceDisplayRowsOptions = {}): TraceEventRow[] {
const orderedEvents = traceEventsForDisplay(events.filter(hasProjectedTraceSequence));
const displayOptions: ResolvedTraceDisplayRowsOptions = {
...options,
sequenceAuthority: traceSequenceAuthority(trace)
};
const orderedEvents = traceEventsForDisplay(
events.filter((event) => hasTraceSequence(event, displayOptions.sequenceAuthority)),
displayOptions.sequenceAuthority
);
const effectiveTrace = traceWithInferredStart(trace, orderedEvents);
const finalResponseText = traceFinalResponseText(effectiveTrace);
const finalResponseFingerprint = traceMessageFingerprint(finalResponseText);
@@ -41,7 +54,7 @@ export function traceDisplayRows(trace: Record<string, unknown> = {}, events: Tr
if (renderedToolIdentities.has(identity)) continue;
renderedToolIdentities.add(identity);
}
rows.push(traceToolCallRow(effectiveTrace, event, options));
rows.push(traceToolCallRow(effectiveTrace, event, displayOptions));
continue;
}
if (isRequestTraceEvent(event)) {
@@ -53,33 +66,37 @@ export function traceDisplayRows(trace: Record<string, unknown> = {}, events: Tr
if (isTerminalAssistantTraceEvent(event)) {
if (hasCompletionEvent) continue;
if (finalResponseText) {
rows.push(traceFinalResponseRow(event, finalResponseText, options));
rows.push(traceFinalResponseRow(event, finalResponseText, displayOptions));
} else {
rows.push(traceAssistantMessageRow(effectiveTrace, event, { terminal: true }, options));
rows.push(traceAssistantMessageRow(effectiveTrace, event, { terminal: true }, displayOptions));
}
continue;
}
if (isAssistantTraceEvent(event)) {
if (finalResponseFingerprint && traceAssistantEventFingerprint(effectiveTrace, event) === finalResponseFingerprint) continue;
rows.push(traceAssistantMessageRow(effectiveTrace, event, { terminal: false }, options));
rows.push(traceAssistantMessageRow(effectiveTrace, event, { terminal: false }, displayOptions));
continue;
}
if (isCompletionTraceEvent(event)) {
completionEvent ??= event;
continue;
}
rows.push(traceDisplayRow(effectiveTrace, event, options));
rows.push(traceDisplayRow(effectiveTrace, event, displayOptions));
}
if (completionEvent) rows.push(traceCompletionSummaryRow(effectiveTrace, completionEvent, options));
const progressRow = traceBackendProgressSummaryRow(effectiveTrace, rows, orderedEvents, options);
if (completionEvent) rows.push(traceCompletionSummaryRow(effectiveTrace, completionEvent, displayOptions));
const progressRow = traceBackendProgressSummaryRow(effectiveTrace, rows, orderedEvents, displayOptions);
if (progressRow) rows.push(progressRow);
if (rows.length > 0) return rows;
if (orderedEvents.length === 0) return [];
return orderedEvents.filter((event) => !isSuppressedTraceEvent(event)).map((event) => traceDisplayRow(effectiveTrace, event, options));
return orderedEvents.filter((event) => !isSuppressedTraceEvent(event)).map((event) => traceDisplayRow(effectiveTrace, event, displayOptions));
}
function hasProjectedTraceSequence(event: TraceEvent | null | undefined): boolean {
return traceEventDisplaySeq(event) !== null;
function traceSequenceAuthority(trace: Record<string, unknown>): TraceSequenceAuthority {
return nonEmptyString(trace.eventSource) === "hwlab-kafka-sse" ? "source" : "projected";
}
function hasTraceSequence(event: TraceEvent | null | undefined, authority: TraceSequenceAuthority): boolean {
return traceEventDisplaySeq(event, authority) !== null;
}
export function renderTraceRowsMarkdown(rows: TraceEventRow[] = []): string {
@@ -106,10 +123,11 @@ function renderTraceMessageRowMarkdown(row: TraceEventRow): string {
return traceBoldMarkdown(row.header || `_No readable trace row body._`);
}
function traceEventsForDisplay(events: TraceEvent[] = []): TraceEvent[] {
function traceEventsForDisplay(events: TraceEvent[] = [], authority: TraceSequenceAuthority): TraceEvent[] {
if (authority === "source") return [...events];
return [...events].sort((left, right) => {
const leftSeq = traceEventDisplaySeq(left);
const rightSeq = traceEventDisplaySeq(right);
const leftSeq = traceEventDisplaySeq(left, authority);
const rightSeq = traceEventDisplaySeq(right, authority);
if (leftSeq !== null || rightSeq !== null) {
if (leftSeq === null) return 1;
if (rightSeq === null) return -1;
@@ -126,12 +144,20 @@ function traceEventsForDisplay(events: TraceEvent[] = []): TraceEvent[] {
});
}
function traceEventDisplaySeq(event: TraceEvent | null | undefined): number | null {
return numberOrNull(event?.projectedSeq);
function traceEventDisplaySeq(event: TraceEvent | null | undefined, authority: TraceSequenceAuthority): number | null {
return numberOrNull(authority === "source" ? event?.sourceSeq : event?.projectedSeq);
}
function traceEventIdentityToken(event: TraceEvent): string | number | null {
const seq = traceEventDisplaySeq(event);
function traceEventIdentityToken(event: TraceEvent, authority: TraceSequenceAuthority): string | number | null {
if (authority === "source") {
const sourceEventId = nonEmptyString(event.sourceEventId);
if (sourceEventId) return sourceEventId;
const runId = nonEmptyString(event.runId);
const sourceSeq = traceEventDisplaySeq(event, authority);
if (runId && sourceSeq !== null) return `${runId}:${sourceSeq}`;
return sourceSeq;
}
const seq = traceEventDisplaySeq(event, authority);
return seq ?? nonEmptyString(event.projectedSeq);
}
@@ -149,15 +175,16 @@ function stripTraceToolOutputFromSummary(value: string): string {
return value.replace(/\s+(stdout:|stderr:|exitCode=).*$/isu, "").trim();
}
function traceToolCallRow(trace: Record<string, unknown>, event: TraceEvent, options: TraceDisplayRowsOptions): TraceEventRow {
function traceToolCallRow(trace: Record<string, unknown>, event: TraceEvent, options: ResolvedTraceDisplayRowsOptions): TraceEventRow {
const command = cleanShellCommand(event.command);
const status = traceStatusToken(event);
const toolName = traceToolName(event, command);
const body = traceToolCallBody(event, command);
const eventKey = traceEventIdentityToken(event);
const eventKey = traceEventIdentityToken(event, options.sequenceAuthority);
return {
rowId: `tool:${event.itemId ?? eventKey ?? `${event.label ?? event.type ?? "tool"}:${event.createdAt ?? "unknown"}`}`,
seq: traceEventDisplaySeq(event),
seq: traceEventDisplaySeq(event, options.sequenceAuthority),
sequenceAuthority: options.sequenceAuthority,
tone: traceEventTone(event),
header: `${traceEventClock(event, options)} total=${formatTraceDuration(traceRelativeMs(trace, event))} ${status} ${toolName}`.trim(),
body,
@@ -221,14 +248,15 @@ export function traceNoiseEventCount(events: TraceEvent[] = []): number {
return Array.isArray(events) ? events.filter((event) => isNoisyTraceEvent(event)).length : 0;
}
function traceAssistantMessageRow(trace: Record<string, unknown>, event: TraceEvent, { terminal }: { terminal: boolean }, options: TraceDisplayRowsOptions): TraceEventRow {
function traceAssistantMessageRow(trace: Record<string, unknown>, event: TraceEvent, { terminal }: { terminal: boolean }, options: ResolvedTraceDisplayRowsOptions): TraceEventRow {
const text = traceAssistantEventText(trace, event);
const index = Number.isInteger(event.messageIndex) ? Number(event.messageIndex) : null;
const count = Number.isInteger(event.messageCount) ? Number(event.messageCount) : null;
const eventKey = traceEventIdentityToken(event);
const eventKey = traceEventIdentityToken(event, options.sequenceAuthority);
return {
rowId: `event:${eventKey ?? `${event.label ?? event.type ?? "assistant"}:${event.createdAt ?? "unknown"}`}`,
seq: traceEventDisplaySeq(event),
seq: traceEventDisplaySeq(event, options.sequenceAuthority),
sequenceAuthority: options.sequenceAuthority,
tone: "ok",
header: `${traceEventClock(event, options)} ${terminal ? "助手最终消息" : "助手消息"}${index ? ` ${index}${count ? `/${count}` : ""}` : ""}`,
terminal: terminal ? true : undefined,
@@ -250,11 +278,12 @@ function traceMessageFingerprint(value: unknown): string | null {
return text || null;
}
function traceFinalResponseRow(event: TraceEvent, finalText: string, options: TraceDisplayRowsOptions): TraceEventRow {
const eventKey = traceEventIdentityToken(event);
function traceFinalResponseRow(event: TraceEvent, finalText: string, options: ResolvedTraceDisplayRowsOptions): TraceEventRow {
const eventKey = traceEventIdentityToken(event, options.sequenceAuthority);
return {
rowId: `trace-final-response:${eventKey ?? "completed"}`,
seq: traceEventDisplaySeq(event),
seq: traceEventDisplaySeq(event, options.sequenceAuthority),
sequenceAuthority: options.sequenceAuthority,
tone: "ok",
header: `${traceEventClock(event, options)} 助手最终消息`,
terminal: true,
@@ -268,24 +297,26 @@ function traceFinalResponseText(trace: Record<string, unknown>): string | null {
return nonEmptyString(response?.text ?? response?.content ?? response?.message);
}
function traceCompletionSummaryRow(trace: Record<string, unknown>, event: TraceEvent, options: TraceDisplayRowsOptions): TraceEventRow {
const eventKey = traceEventIdentityToken(event);
function traceCompletionSummaryRow(trace: Record<string, unknown>, event: TraceEvent, options: ResolvedTraceDisplayRowsOptions): TraceEventRow {
const eventKey = traceEventIdentityToken(event, options.sequenceAuthority);
return {
rowId: `trace-completion:${eventKey ?? "turn"}`,
seq: traceEventDisplaySeq(event),
seq: traceEventDisplaySeq(event, options.sequenceAuthority),
sequenceAuthority: options.sequenceAuthority,
tone: traceEventTone(event),
header: `${traceEventClock(event, options)} 轮次完成(总耗时 ${formatTraceDuration(traceRelativeMs(trace, event))}`,
body: null
};
}
function traceNoiseSummaryRow(trace: Record<string, unknown>, events: TraceEvent[], options: TraceDisplayRowsOptions): TraceEventRow {
function traceNoiseSummaryRow(trace: Record<string, unknown>, events: TraceEvent[], options: ResolvedTraceDisplayRowsOptions): TraceEventRow {
const lastEvent = events.at(-1);
const status = nonEmptyString(trace.status ?? trace.traceStatus) ?? "running";
const lastLabel = lastEvent ? readableTraceLabel(lastEvent) : "未观测";
return {
rowId: "trace-noise-summary",
seq: traceEventDisplaySeq(lastEvent),
seq: traceEventDisplaySeq(lastEvent, options.sequenceAuthority),
sequenceAuthority: options.sequenceAuthority,
tone: lastEvent ? traceEventTone(lastEvent) : "source",
header: `Trace ${status},等待可读事件`,
body: `已隐藏 ${events.length} 条 AgentRun backend 状态事件。最新原始事件:${lastLabel}`,
@@ -293,23 +324,24 @@ function traceNoiseSummaryRow(trace: Record<string, unknown>, events: TraceEvent
};
}
function traceBackendProgressSummaryRow(trace: Record<string, unknown>, rows: TraceEventRow[], events: TraceEvent[], options: TraceDisplayRowsOptions): TraceEventRow | null {
function traceBackendProgressSummaryRow(trace: Record<string, unknown>, rows: TraceEventRow[], events: TraceEvent[], options: ResolvedTraceDisplayRowsOptions): TraceEventRow | null {
if (!Array.isArray(events) || events.length === 0) return null;
if (rows.some((row) => row.rowId.startsWith("tool:") || /|/u.test(row.header))) return null;
const lastRenderedSeq = Math.max(0, ...rows.map((row) => row.seq ?? 0).filter(Number.isFinite));
const hiddenProgress = events.filter((event) => {
const seq = traceEventDisplaySeq(event) ?? 0;
const seq = traceEventDisplaySeq(event, options.sequenceAuthority) ?? 0;
return seq > lastRenderedSeq && (isRequestTraceEvent(event) || isSetupTraceEvent(event) || isNoisyTraceEvent(event));
});
return hiddenProgress.length > 0 ? traceNoiseSummaryRow(trace, hiddenProgress, options) : null;
}
function traceDisplayRow(trace: Record<string, unknown>, event: TraceEvent, options: TraceDisplayRowsOptions): TraceEventRow {
function traceDisplayRow(trace: Record<string, unknown>, event: TraceEvent, options: ResolvedTraceDisplayRowsOptions): TraceEventRow {
const label = readableTraceLabel(event);
const eventKey = traceEventIdentityToken(event);
const eventKey = traceEventIdentityToken(event, options.sequenceAuthority);
return {
rowId: `event:${eventKey ?? `${event.label ?? event.type ?? "event"}:${event.createdAt ?? "unknown"}`}`,
seq: traceEventDisplaySeq(event),
seq: traceEventDisplaySeq(event, options.sequenceAuthority),
sequenceAuthority: options.sequenceAuthority,
tone: traceEventTone(event),
header: `${traceEventClock(event, options)} total=${formatTraceDuration(traceRelativeMs(trace, event))} ${traceStatusToken(event)} ${label}`.trim(),
body: traceDisplayBody(event)
@@ -380,8 +412,12 @@ function toolIdentity(event: TraceEvent): string | null {
function traceSourceEventKey(event: TraceEvent | null | undefined): string | null {
if (!event || typeof event !== "object") return null;
const source = nonEmptyString(event.source);
const sourceEventId = nonEmptyString(event.sourceEventId);
if (sourceEventId) return `source-event:${sourceEventId}`;
const runId = nonEmptyString(event.runId);
const sourceSeq = nonEmptyString(event.sourceSeq);
if (runId && sourceSeq) return `run-source:${runId}:${sourceSeq}`;
const source = nonEmptyString(event.source);
if (!source || !sourceSeq) return null;
return `${source}:${sourceSeq}:${nonEmptyString(event.label ?? event.type) ?? "event"}`;
}
@@ -102,7 +102,7 @@ const terminalAssistantFinalText = [
].join("\n");
const canonicalTraceFinalText = "messageProjection 的 sealed final response 才是主消息正文。";
const traceDetailConflictText = "TRACE_DETAIL_CONFLICT_SHOULD_NOT_REPLACE_MESSAGE";
const runtimeConfigScript = `<script>\nwindow.HWLAB_CLOUD_WEB_CONFIG = {\n ...(window.HWLAB_CLOUD_WEB_CONFIG ?? {}),\n displayTime: window.HWLAB_CLOUD_WEB_CONFIG?.displayTime ?? { timeZone: "Asia/Shanghai", locale: "zh-CN", label: "北京时间" },\n workbench: {\n ...(window.HWLAB_CLOUD_WEB_CONFIG?.workbench ?? {}),\n realtimeFeatures: window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.realtimeFeatures ?? { liveKafkaSse: false, projectionRealtime: true },\n debugCapabilities: window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.debugCapabilities ?? { isolatedKafka: false },\n traceTimeline: window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.traceTimeline ?? { autoExpandRunning: false, autoCollapseTerminal: false }\n }\n};\n</script>`;
const runtimeConfigScript = `<script>\nwindow.HWLAB_CLOUD_WEB_CONFIG = {\n ...(window.HWLAB_CLOUD_WEB_CONFIG ?? {}),\n displayTime: window.HWLAB_CLOUD_WEB_CONFIG?.displayTime ?? { timeZone: "Asia/Shanghai", locale: "zh-CN", label: "北京时间" },\n workbench: {\n ...(window.HWLAB_CLOUD_WEB_CONFIG?.workbench ?? {}),\n realtimeFeatures: window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.realtimeFeatures ?? { liveKafkaSse: false, projectionRealtime: true },\n debugCapabilities: window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.debugCapabilities ?? { isolatedKafka: false, rawHwlabEventWindow: { enabled: false, maxEntries: 1, maxRetainedBytes: 1 } },\n traceTimeline: window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.traceTimeline ?? { autoExpandRunning: false, autoCollapseTerminal: false }\n }\n};\n</script>`;
let state = createScenarioState("baseline");
const sseClients = new Set<ServerResponse>();
@@ -117,6 +117,24 @@ test("R1 shared trace renderer keeps UTC by default and lets Web supply display
assert.match(displayRows[0]?.header ?? "", /^14:34:14 /u);
});
test("pure Kafka Trace renders sourceSeq-only rows in ingress order without projectedSeq", () => {
const events = [
{ sourceEventId: "evt_tool", runId: "run_live", source: "agentrun.kafka", sourceSeq: 8, label: "agentrun:tool:commandExecution", type: "tool_call", toolName: "commandExecution", status: "completed", command: "make test" },
{ sourceEventId: "evt_assistant", runId: "run_live", source: "agentrun.kafka", sourceSeq: 7, label: "agentrun:assistant:message", type: "assistant", status: "running", message: "live answer" },
{ sourceEventId: "evt_assistant", runId: "run_live", source: "agentrun.kafka", sourceSeq: 70, label: "agentrun:assistant:message", type: "assistant", status: "running", message: "duplicate should not render" },
{ sourceEventId: "evt_terminal", runId: "run_live", source: "agentrun.kafka", sourceSeq: 9, label: "agentrun:terminal:completed", type: "result", status: "completed", terminal: true }
];
const liveRows = traceDisplayRows({ traceId: "trc_live_source", eventSource: "hwlab-kafka-sse", status: "completed" }, events);
assert.deepEqual(liveRows.map((row) => row.seq), [8, 7, 9]);
assert.ok(liveRows.every((row) => row.sequenceAuthority === "source"));
assert.equal(liveRows.some((row) => row.body === "duplicate should not render"), false);
assert.equal(events.some((event) => "projectedSeq" in event), false);
const durableRows = traceDisplayRows({ traceId: "trc_durable", eventSource: "trace-api", status: "completed" }, events);
assert.deepEqual(durableRows, []);
});
test("R1 trace API snapshots stay authoritative over compact result traces", () => {
const previous = {
traceId: "trc_merge",
@@ -28,6 +28,19 @@ test("isolated Kafka debug replay path is trace-scoped and explicit", () => {
);
});
test("isolated Kafka debug v2 path carries replay correlation and output barrier", () => {
assert.equal(
workbenchKafkaSseDebugPath({
stream: "hwlab-debug",
fromBeginning: true,
traceId: "trc_isolated_v2",
replayId: "rpl_isolated_v2",
outputBarrier: { partition: 0, firstOffset: "42", lastOffset: "51" }
}),
"/v1/workbench/debug/kafka-sse/events?stream=hwlab-debug&fromBeginning=true&traceId=trc_isolated_v2&replayId=rpl_isolated_v2&partition=0&firstOffset=42&lastOffset=51"
);
});
test("Kafka SSE debug correlation ids normalize stdio snake_case metadata", () => {
assert.deepEqual(
workbenchKafkaSseDebugCorrelationIds({
+70 -11
View File
@@ -52,6 +52,44 @@ export interface WorkbenchDebugFakeSseStream {
export type WorkbenchKafkaSseDebugStreamName = "stdio" | "agentrun" | "hwlab" | "hwlab-debug";
export interface WorkbenchKafkaDebugOffsetBarrier {
partition: number;
firstOffset: string;
lastOffset: string;
}
export interface WorkbenchKafkaDebugReplayCounts {
scanned?: number;
barrierScanned?: number;
parsed?: number;
traceMatched?: number;
replayMatched?: number;
matched?: number;
delivered?: number;
clientReceived?: number | null;
decoded?: number | null;
applied?: number | null;
}
export interface WorkbenchKafkaDebugReplayResult {
contractVersion?: string;
replayId?: string | null;
traceId?: string | null;
phase?: string;
code?: string;
ok?: boolean;
message?: string;
terminalObserved?: boolean;
counts?: WorkbenchKafkaDebugReplayCounts;
offsetRange?: Record<string, unknown> | null;
[key: string]: unknown;
}
export interface WorkbenchKafkaSseDebugIngressFrame {
eventName: string;
decodeStatus: "accepted" | "invalid-json" | "not-object";
}
export interface WorkbenchKafkaSseDebugEvent {
ok?: boolean;
contractVersion?: string;
@@ -66,6 +104,12 @@ export interface WorkbenchKafkaSseDebugEvent {
limit?: number;
timeoutMs?: number;
terminalObserved?: boolean;
replayId?: string | null;
phase?: string;
code?: string;
result?: WorkbenchKafkaDebugReplayResult | null;
counts?: WorkbenchKafkaDebugReplayCounts;
offsetRange?: Record<string, unknown> | null;
traceId?: string | null;
groupId?: string;
stream?: WorkbenchKafkaSseDebugStreamName;
@@ -89,6 +133,9 @@ export interface WorkbenchKafkaSseDebugStreamOptions {
sessionId?: string | null;
runId?: string | null;
commandId?: string | null;
replayId?: string | null;
outputBarrier?: WorkbenchKafkaDebugOffsetBarrier | null;
onIngress?: (frame: WorkbenchKafkaSseDebugIngressFrame) => void;
onEvent: (event: WorkbenchKafkaSseDebugEvent, eventName: string) => void;
onOpen?: () => void;
onError?: (event: Event) => void;
@@ -149,18 +196,13 @@ export function connectWorkbenchKafkaSseDebug(options: WorkbenchKafkaSseDebugStr
source.onerror = (event) => options.onError?.(event);
const listeners = DEBUG_KAFKA_SSE_EVENTS.map((name) => {
const listener = (event: MessageEvent) => {
const payload = parseKafkaSseDebugEvent(event.data);
if (!payload) return;
options.onEvent(payload, name);
if (name === "hwlab.kafka.connected" && payload.consumerReady === true) options.onOpen?.();
if (name === "hwlab.kafka.error") options.onError?.(event);
deliverKafkaSseDebugFrame(event.data, name, event, options);
};
source.addEventListener(name, listener);
return { name, listener };
});
source.onmessage = (event) => {
const payload = parseKafkaSseDebugEvent(event.data);
if (payload) options.onEvent(payload, "message");
deliverKafkaSseDebugFrame(event.data, "message", event, options);
};
return {
close() {
@@ -170,13 +212,19 @@ export function connectWorkbenchKafkaSseDebug(options: WorkbenchKafkaSseDebugStr
};
}
export function workbenchKafkaSseDebugPath(options: Pick<WorkbenchKafkaSseDebugStreamOptions, "stream" | "fromBeginning" | "traceId" | "sessionId" | "runId" | "commandId">): string {
export function workbenchKafkaSseDebugPath(options: Pick<WorkbenchKafkaSseDebugStreamOptions, "stream" | "fromBeginning" | "traceId" | "sessionId" | "runId" | "commandId" | "replayId" | "outputBarrier">): string {
const query = new URLSearchParams({ stream: options.stream });
if (options.fromBeginning === true) query.set("fromBeginning", "true");
appendDebugFilter(query, "traceId", options.traceId);
appendDebugFilter(query, "sessionId", options.sessionId);
appendDebugFilter(query, "runId", options.runId);
appendDebugFilter(query, "commandId", options.commandId);
appendDebugFilter(query, "replayId", options.replayId);
if (options.outputBarrier) {
query.set("partition", String(options.outputBarrier.partition));
query.set("firstOffset", options.outputBarrier.firstOffset);
query.set("lastOffset", options.outputBarrier.lastOffset);
}
return `/v1/workbench/debug/kafka-sse/events?${query.toString()}`;
}
@@ -214,12 +262,23 @@ function parseRealtimeEvent(raw: string): WorkbenchRealtimeEvent | null {
}
}
function parseKafkaSseDebugEvent(raw: string): WorkbenchKafkaSseDebugEvent | null {
function deliverKafkaSseDebugFrame(raw: unknown, eventName: string, transportEvent: MessageEvent, options: WorkbenchKafkaSseDebugStreamOptions): void {
const decoded = parseKafkaSseDebugEvent(typeof raw === "string" ? raw : String(raw ?? ""));
options.onIngress?.({ eventName, decodeStatus: decoded.status });
if (!decoded.payload) return;
options.onEvent(decoded.payload, eventName);
if (eventName === "hwlab.kafka.connected" && decoded.payload.consumerReady === true) options.onOpen?.();
if (eventName === "hwlab.kafka.error") options.onError?.(transportEvent);
}
function parseKafkaSseDebugEvent(raw: string): { payload: WorkbenchKafkaSseDebugEvent | null; status: WorkbenchKafkaSseDebugIngressFrame["decodeStatus"] } {
try {
const value = JSON.parse(raw);
return value && typeof value === "object" && !Array.isArray(value) ? value as WorkbenchKafkaSseDebugEvent : null;
return value && typeof value === "object" && !Array.isArray(value)
? { payload: value as WorkbenchKafkaSseDebugEvent, status: "accepted" }
: { payload: null, status: "not-object" };
} catch {
return null;
return { payload: null, status: "invalid-json" };
}
}
@@ -1,7 +1,7 @@
import assert from "node:assert/strict";
import { test } from "bun:test";
import { realtimeCoalesceKey, type WorkbenchRealtimeEvent } from "./workbench-events";
import { connectWorkbenchEvents, realtimeCoalesceKey, type WorkbenchRealtimeEvent, type WorkbenchSseIngressFrame } from "./workbench-events";
import { createCoalescedEventQueue } from "../utils/scheduler/coalesced-event-queue";
test("turn snapshot coalescing keys by trace instead of per sequence", () => {
@@ -39,3 +39,66 @@ test("live Kafka envelopes from one trace retain every assistant, tool, output,
assert.deepEqual(delivered.map((event) => event.sourceEventId), ["evt_assistant", "evt_tool", "evt_output", "evt_terminal"]);
});
test("product EventSource tees every raw HWLAB frame once before decode and reducer delivery", async () => {
const originalEventSource = globalThis.EventSource;
const ingress: WorkbenchSseIngressFrame[] = [];
const delivered: WorkbenchRealtimeEvent[] = [];
FakeProductEventSource.instances = [];
globalThis.EventSource = FakeProductEventSource as unknown as typeof EventSource;
try {
const stream = connectWorkbenchEvents({
realtimeCapabilities: { liveKafkaSse: true, projectionRealtime: false },
sessionId: "ses_raw_tee",
flushYieldMs: 1,
onIngress: (frame) => ingress.push(frame),
onEvent: (event) => delivered.push(event)
});
assert.ok(stream);
const source = FakeProductEventSource.instances[0];
assert.ok(source);
source.emitRaw("hwlab.event.v1", '{"schema":"hwlab.event.v1","eventId":"evt_raw_tee"}');
source.emitRaw("hwlab.event.v1", "{invalid-json");
await new Promise((resolve) => setTimeout(resolve, 10));
assert.equal(FakeProductEventSource.instances.length, 1);
assert.deepEqual(ingress.map((frame) => [frame.eventName, frame.decodeStatus]), [
["hwlab.event.v1", "accepted"],
["hwlab.event.v1", "invalid-json"]
]);
assert.equal(delivered.length, 1);
assert.equal(delivered[0]?.eventId, "evt_raw_tee");
stream.close();
} finally {
globalThis.EventSource = originalEventSource;
}
});
class FakeProductEventSource {
static instances: FakeProductEventSource[] = [];
onopen: ((event: Event) => void) | null = null;
onerror: ((event: Event) => void) | null = null;
onmessage: ((event: MessageEvent) => void) | null = null;
private readonly listeners = new Map<string, Set<(event: MessageEvent) => void>>();
constructor(readonly url: string | URL) {
FakeProductEventSource.instances.push(this);
}
addEventListener(name: string, listener: (event: MessageEvent) => void): void {
const listeners = this.listeners.get(name) ?? new Set();
listeners.add(listener);
this.listeners.set(name, listeners);
}
removeEventListener(name: string, listener: (event: MessageEvent) => void): void {
this.listeners.get(name)?.delete(listener);
}
emitRaw(name: string, rawText: string): void {
const event = { data: rawText } as MessageEvent;
for (const listener of this.listeners.get(name) ?? []) listener(event);
}
close(): void {}
}
@@ -107,6 +107,7 @@ export interface WorkbenchEventStreamOptions {
flushMaxItemsPerChunk?: number | null;
flushMaxChunkMs?: number | null;
flushYieldMs?: number | null;
onIngress?: (frame: WorkbenchSseIngressFrame) => void;
onEvent: (event: WorkbenchRealtimeEvent, eventName: string) => void;
onOpen?: () => void;
onError?: (event: Event) => void;
@@ -116,6 +117,12 @@ export interface WorkbenchEventStream {
close: () => void;
}
export interface WorkbenchSseIngressFrame {
eventName: string;
rawText: string;
decodeStatus: "accepted" | "invalid-json" | "not-object";
}
interface QueuedRealtimeEvent {
payload: WorkbenchRealtimeEvent;
eventName: string;
@@ -182,15 +189,13 @@ export function connectWorkbenchEvents(options: WorkbenchEventStreamOptions): Wo
};
const listeners = WORKBENCH_EVENT_NAMES.map((name) => {
const listener = (event: MessageEvent) => {
const payload = parseRealtimeEvent(event.data);
if (payload) queue.push({ payload, eventName: name });
enqueueRealtimeFrame(event.data, name, options, queue);
};
source.addEventListener(name, listener);
return { name, listener };
});
source.onmessage = (event) => {
const payload = parseRealtimeEvent(event.data);
if (payload) queue.push({ payload, eventName: "message" });
enqueueRealtimeFrame(event.data, "message", options, queue);
};
return {
close() {
@@ -244,12 +249,21 @@ function appendNumberParam(params: URLSearchParams, key: string, value: number |
params.set(key, String(Math.trunc(parsed)));
}
function parseRealtimeEvent(raw: string): WorkbenchRealtimeEvent | null {
function enqueueRealtimeFrame(raw: unknown, eventName: string, options: WorkbenchEventStreamOptions, queue: ReturnType<typeof createCoalescedEventQueue<QueuedRealtimeEvent>>): void {
const rawText = typeof raw === "string" ? raw : String(raw ?? "");
const decoded = decodeRealtimeEvent(rawText);
options.onIngress?.({ eventName, rawText, decodeStatus: decoded.status });
if (decoded.payload) queue.push({ payload: decoded.payload, eventName });
}
function decodeRealtimeEvent(raw: string): { payload: WorkbenchRealtimeEvent | null; status: WorkbenchSseIngressFrame["decodeStatus"] } {
try {
const value = JSON.parse(raw);
return value && typeof value === "object" ? value as WorkbenchRealtimeEvent : null;
return value && typeof value === "object" && !Array.isArray(value)
? { payload: value as WorkbenchRealtimeEvent, status: "accepted" }
: { payload: null, status: "not-object" };
} catch {
return null;
return { payload: null, status: "invalid-json" };
}
}
@@ -0,0 +1,77 @@
// @vitest-environment jsdom
import { beforeEach, describe, expect, it } from "vitest";
import { mount } from "@vue/test-utils";
import { nextTick } from "vue";
import TraceTimeline from "./TraceTimeline.vue";
describe("TraceTimeline sequence authority", () => {
beforeEach(() => {
window.HWLAB_CLOUD_WEB_CONFIG = {
displayTime: { timeZone: "Asia/Shanghai", locale: "zh-CN", label: "北京时间" }
};
window.localStorage.clear();
globalThis.ResizeObserver = FakeResizeObserver as unknown as typeof ResizeObserver;
Object.defineProperty(HTMLElement.prototype, "scrollTo", { configurable: true, value: () => undefined });
});
it("renders sourceSeq-only live Kafka rows while terminal and does not label them projected", async () => {
const wrapper = mount(TraceTimeline, {
props: {
autoExpanded: true,
trace: {
traceId: "trc_live_timeline",
eventSource: "hwlab-kafka-sse",
status: "completed",
eventCount: 2,
events: [
{ sourceEventId: "evt_live_answer", runId: "run_live", sourceSeq: 4, source: "agentrun.kafka", type: "assistant", label: "agentrun:assistant:message", message: "live answer" },
{ sourceEventId: "evt_live_terminal", runId: "run_live", sourceSeq: 5, source: "agentrun.kafka", type: "result", label: "agentrun:terminal:completed", status: "completed", terminal: true }
]
}
}
});
await nextTick();
const disclosure = wrapper.get(".trace-disclosure");
expect(disclosure.attributes("data-event-count")).toBe("2");
expect(Number(disclosure.attributes("data-readable-row-count"))).toBeGreaterThan(0);
const rows = wrapper.findAll('[data-testid="trace-render-row"]');
expect(rows.length).toBeGreaterThan(0);
expect(rows[0]?.attributes("data-event-seq-authority")).toBe("source");
expect(rows[0]?.attributes("data-event-seq")).toBe("4");
expect(rows[0]?.attributes("data-projected-seq")).toBeUndefined();
expect(wrapper.text()).toContain("live answer");
expect(wrapper.text()).not.toContain("运行已完成,暂无可读 Trace");
wrapper.unmount();
});
it("keeps durable Trace rows on projected sequence authority", async () => {
const wrapper = mount(TraceTimeline, {
props: {
autoExpanded: true,
trace: {
traceId: "trc_projected_timeline",
eventSource: "trace-api",
status: "running",
eventCount: 1,
events: [{ projectedSeq: 8, type: "assistant", label: "agentrun:assistant:message", message: "projected answer" }]
}
}
});
await nextTick();
const row = wrapper.get('[data-testid="trace-render-row"]');
expect(row.attributes("data-event-seq-authority")).toBe("projected");
expect(row.attributes("data-event-seq")).toBe("8");
expect(row.attributes("data-projected-seq")).toBe("8");
wrapper.unmount();
});
});
class FakeResizeObserver {
observe(): void {}
unobserve(): void {}
disconnect(): void {}
}
@@ -51,7 +51,7 @@ watch(() => [props.trace?.traceId, props.storageKey, props.defaultExpanded, prop
}, { immediate: true });
function rowKey(row: TraceEventRow, index: number): string {
return `${row.rowId}-${row.seq ?? "no-seq"}-${index}`;
return `${row.rowId}-${row.sequenceAuthority}-${row.seq ?? "no-seq"}-${index}`;
}
function onDisclosureToggle(event: Event): void {
@@ -144,7 +144,7 @@ function firstNonEmptyString(...values: unknown[]): string | null {
<div v-if="expanded" class="trace-disclosure-body">
<p v-if="hiddenRowCount > 0" class="trace-window-notice">显示最近 {{ renderedRows.length }} / {{ readableRows.length }} </p>
<ol v-if="renderedRows.length > 0" ref="listRef" @scroll="onScroll">
<li v-for="(row, index) in renderedRows" :key="rowKey(row, index)" class="trace-render-row" data-testid="trace-render-row" :data-trace-id="trace.traceId || undefined" :data-projected-seq="row.seq ?? undefined" :data-event-kind="row.rowId" :data-event-status="row.tone" :data-terminal="row.terminal ? 'true' : 'false'" :data-row-kind="rowIsTool(row) ? 'tool' : 'message'" :data-row-id="row.rowId" :aria-posinset="row.seq ?? hiddenRowCount + index + 1">
<li v-for="(row, index) in renderedRows" :key="rowKey(row, index)" class="trace-render-row" data-testid="trace-render-row" :data-trace-id="trace.traceId || undefined" :data-event-seq="row.seq ?? undefined" :data-event-seq-authority="row.sequenceAuthority" :data-projected-seq="row.sequenceAuthority === 'projected' ? row.seq ?? undefined : undefined" :data-event-kind="row.rowId" :data-event-status="row.tone" :data-terminal="row.terminal ? 'true' : 'false'" :data-row-kind="rowIsTool(row) ? 'tool' : 'message'" :data-row-id="row.rowId" :aria-posinset="hiddenRowCount + index + 1">
<details v-if="rowIsTool(row)" class="trace-tool-details">
<summary>
<code>{{ row.header }}</code>
@@ -1,6 +1,6 @@
// @vitest-environment jsdom
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { mount } from "@vue/test-utils";
import { nextTick } from "vue";
@@ -19,23 +19,46 @@ describe("WorkbenchKafkaDebugPanel", () => {
});
it("replays only after an explicit click, completes on terminal, and clears locally", async () => {
const wrapper = mount(WorkbenchKafkaDebugPanel, { props: { traceId: "trc_component_debug" } });
const wrapper = mount(WorkbenchKafkaDebugPanel, {
props: {
traceId: "trc_component_debug",
replayId: "rpl_component_debug",
outputBarrier: { partition: 0, firstOffset: "10", lastOffset: "11" }
}
});
expect(FakeEventSource.instances).toHaveLength(0);
await wrapper.get('[data-testid="workbench-kafka-debug-replay"]').trigger("click");
const source = FakeEventSource.instances[0];
const source = FakeEventSource.instances[0]!;
expect(source.url).toContain("stream=hwlab-debug");
expect(source.url).toContain("fromBeginning=true");
expect(source.url).toContain("traceId=trc_component_debug");
expect(source.url).toContain("replayId=rpl_component_debug");
expect(source.url).toContain("partition=0");
expect(source.url).toContain("firstOffset=10");
expect(source.url).toContain("lastOffset=11");
source.emit("hwlab.kafka.connected", { consumerReady: true, debugIsolation: true, deliverySemantics: "debug-replay", liveOnly: false, replay: true, topic: "hwlab.event.debug.v1", groupId: "debug-group-unique" });
source.emit("hwlab.kafka.event", debugRecord("assistant", { type: "assistant", assistantText: "debug answer", status: "running" }));
source.emit("hwlab.kafka.event", debugRecord("terminal", { type: "result", eventType: "terminal", terminal: true, status: "completed" }));
source.emit("hwlab.kafka.replay-complete", { reason: "terminal", terminalObserved: true, count: 2 });
source.emit("hwlab.kafka.replay-complete", {
reason: "terminal",
terminalObserved: true,
count: 2,
result: {
contractVersion: "workbench-kafka-debug-replay-v2",
replayId: "rpl_component_debug",
code: "terminal_complete",
terminalObserved: true,
counts: { scanned: 2, matched: 2, delivered: 2, clientReceived: null, decoded: null, applied: null }
}
});
await nextTick();
expect(wrapper.get('[data-testid="workbench-kafka-debug-panel"]').attributes("data-replay-status")).toBe("completed");
expect(wrapper.get('[data-testid="workbench-kafka-debug-counts"]').text()).toContain("2/2");
expect(wrapper.get('[data-testid="workbench-kafka-debug-layer-counts"]').text()).toContain("scanned=2");
expect(wrapper.get('[data-testid="workbench-kafka-debug-layer-counts"]').text()).toContain("received=2 · decoded=2 · applied=2");
expect(wrapper.get(".message-card").attributes("data-status")).toBe("completed");
expect(wrapper.get(".message-card").text()).toContain("debug answer");
expect(source.closed).toBe(true);
@@ -45,7 +68,7 @@ describe("WorkbenchKafkaDebugPanel", () => {
expect(wrapper.get('[data-testid="workbench-kafka-debug-counts"]').text()).toContain("0/0");
await wrapper.get('[data-testid="workbench-kafka-debug-replay"]').trigger("click");
const incompleteSource = FakeEventSource.instances[1];
const incompleteSource = FakeEventSource.instances[1]!;
incompleteSource.emit("hwlab.kafka.connected", { consumerReady: true, debugIsolation: true, deliverySemantics: "debug-replay", liveOnly: false, replay: true });
incompleteSource.emit("hwlab.kafka.replay-complete", { reason: "timeout", terminalObserved: false, count: 0 });
await nextTick();
@@ -53,6 +76,42 @@ describe("WorkbenchKafkaDebugPanel", () => {
expect(wrapper.text()).toContain("重放未观测到 terminaltimeout");
expect(incompleteSource.closed).toBe(true);
});
it("shows the product raw HWLAB Event buffer without opening another EventSource", async () => {
const firstRaw = '{"schema":"hwlab.event.v1","eventId":"evt_raw_1"}';
const invalidRaw = "{invalid-json";
const clearRawIngress = vi.fn();
const wrapper = mount(WorkbenchKafkaDebugPanel, {
props: {
traceId: "trc_component_raw",
replayEnabled: true,
rawEnabled: true,
onClearRawIngress: clearRawIngress,
rawIngress: {
scopeKey: "ses_component_raw",
receivedCount: 2,
decodedCount: 1,
rejectedCount: 1,
evictedCount: 0,
retainedBytes: firstRaw.length + invalidRaw.length,
rows: [
{ arrivalSeq: 1, receivedAt: "2026-07-10T00:00:00.000Z", eventName: "hwlab.event.v1", rawText: firstRaw, decodeStatus: "accepted", byteLength: firstRaw.length },
{ arrivalSeq: 2, receivedAt: "2026-07-10T00:00:01.000Z", eventName: "hwlab.event.v1", rawText: invalidRaw, decodeStatus: "invalid-json", byteLength: invalidRaw.length }
]
}
}
});
expect(FakeEventSource.instances).toHaveLength(0);
await wrapper.get('[data-testid="workbench-kafka-debug-tab-raw"]').trigger("click");
expect(FakeEventSource.instances).toHaveLength(0);
expect(wrapper.get('[data-testid="workbench-raw-hwlab-counts"]').text()).toContain("received=2");
expect((wrapper.get('[data-testid="workbench-raw-hwlab-text"]').element as HTMLTextAreaElement).value).toBe(`${firstRaw}\n${invalidRaw}`);
await wrapper.get('[data-testid="workbench-raw-hwlab-clear"]').trigger("click");
expect(clearRawIngress).toHaveBeenCalledTimes(1);
expect(FakeEventSource.instances).toHaveLength(0);
});
});
class FakeEventSource {
@@ -2,31 +2,66 @@
// Responsibility: admin-only, local-state view of the YAML-gated hwlab.event.debug.v1 SSE stream.
import { computed, onBeforeUnmount, ref, watch } from "vue";
import { connectWorkbenchKafkaSseDebug, type WorkbenchKafkaSseDebugEvent, type WorkbenchKafkaSseDebugStream } from "@/api/workbench-debug";
import { connectWorkbenchKafkaSseDebug, type WorkbenchKafkaDebugOffsetBarrier, type WorkbenchKafkaDebugReplayCounts, type WorkbenchKafkaDebugReplayResult, type WorkbenchKafkaSseDebugEvent, type WorkbenchKafkaSseDebugStream } from "@/api/workbench-debug";
import StatusBadge from "@/components/common/StatusBadge.vue";
import { applyWorkbenchIsolatedKafkaDebugEvent, createWorkbenchIsolatedKafkaDebugState } from "@/stores/workbench-isolated-kafka-debug";
import { createRawHwlabIngressState, type RawHwlabIngressState } from "@/stores/workbench-raw-hwlab-ingress";
import WorkbenchMessageCard from "./WorkbenchMessageCard.vue";
const props = defineProps<{ traceId: string | null }>();
const props = withDefaults(defineProps<{
traceId: string | null;
replayEnabled?: boolean;
rawEnabled?: boolean;
rawIngress?: RawHwlabIngressState;
onClearRawIngress?: () => void;
replayId?: string | null;
outputBarrier?: WorkbenchKafkaDebugOffsetBarrier | null;
}>(), {
replayEnabled: true,
rawEnabled: false,
rawIngress: () => createRawHwlabIngressState(),
onClearRawIngress: () => undefined,
replayId: null,
outputBarrier: null
});
type InspectorTab = "trace" | "raw";
const state = ref(createWorkbenchIsolatedKafkaDebugState());
const connection = ref<WorkbenchKafkaSseDebugEvent | null>(null);
const status = ref<"idle" | "connecting" | "replaying" | "completed" | "incomplete" | "error" | "closed">("idle");
const transportError = ref<string | null>(null);
const activeTab = ref<InspectorTab>(props.replayEnabled ? "trace" : "raw");
const clientReceived = ref(0);
const clientDecoded = ref(0);
const replayResult = ref<WorkbenchKafkaDebugReplayResult | null>(null);
let stream: WorkbenchKafkaSseDebugStream | null = null;
const badgeStatus = computed(() => status.value === "completed" ? "completed" : ["incomplete", "error"].includes(status.value) ? "failed" : "running");
const statusLabel = computed(() => status.value === "completed" ? "重放完成" : status.value === "incomplete" ? "重放不完整" : status.value === "replaying" ? "重放中" : status.value === "idle" ? "等待重放" : `debug ${status.value}`);
const visibleError = computed(() => transportError.value ?? state.value.error);
const replayBusy = computed(() => ["connecting", "replaying"].includes(status.value));
const rawText = computed(() => props.rawIngress.rows.map((row) => row.rawText).join("\n"));
const replayIdLabel = computed(() => replayResult.value?.replayId ?? connection.value?.replayId ?? props.replayId ?? "-");
const outputBarrierLabel = computed(() => props.outputBarrier
? `${props.outputBarrier.partition}:${props.outputBarrier.firstOffset}-${props.outputBarrier.lastOffset}`
: "-");
const serverCounts = computed<WorkbenchKafkaDebugReplayCounts>(() => replayResult.value?.counts ?? {});
watch(() => props.traceId, resetForTrace);
watch(() => [props.replayEnabled, props.rawEnabled] as const, ([replayEnabled, rawEnabled]) => {
if (activeTab.value === "trace" && !replayEnabled && rawEnabled) activeTab.value = "raw";
if (activeTab.value === "raw" && !rawEnabled && replayEnabled) activeTab.value = "trace";
}, { immediate: true });
onBeforeUnmount(closeStream);
function replayCurrentTrace(): void {
if (!props.replayEnabled) return;
closeStream(false);
state.value = createWorkbenchIsolatedKafkaDebugState();
connection.value = null;
transportError.value = null;
clientReceived.value = 0;
clientDecoded.value = 0;
replayResult.value = null;
const traceId = cleanTraceId(props.traceId);
if (!traceId) {
status.value = "idle";
@@ -37,6 +72,13 @@ function replayCurrentTrace(): void {
stream: "hwlab-debug",
traceId,
fromBeginning: true,
replayId: props.replayId,
outputBarrier: props.outputBarrier,
onIngress: (frame) => {
if (frame.eventName !== "hwlab.kafka.event") return;
clientReceived.value += 1;
if (frame.decodeStatus === "accepted") clientDecoded.value += 1;
},
onOpen: () => { status.value = "replaying"; },
onError: () => {
if (["completed", "incomplete"].includes(status.value)) return;
@@ -62,9 +104,16 @@ function replayCurrentTrace(): void {
if (eventName === "hwlab.kafka.replay-complete") {
stream?.close();
stream = null;
const complete = event.reason === "terminal" && event.terminalObserved === true;
replayResult.value = event.result ?? null;
const complete = event.result
? event.result.code === "terminal_complete" && event.result.terminalObserved === true
: event.reason === "terminal" && event.terminalObserved === true;
status.value = complete ? "completed" : "incomplete";
transportError.value = complete ? null : `重放未观测到 terminal${event.reason || "unknown"}`;
transportError.value = complete
? null
: event.result?.code
? `${event.result.code}: ${event.result.message || "隔离重放未完成"}`
: `重放未观测到 terminal${event.reason || "unknown"}`;
return;
}
if (eventName !== "hwlab.kafka.event") return;
@@ -88,6 +137,9 @@ function clearPanel(): void {
state.value = createWorkbenchIsolatedKafkaDebugState();
connection.value = null;
transportError.value = null;
clientReceived.value = 0;
clientDecoded.value = 0;
replayResult.value = null;
status.value = "idle";
}
@@ -106,25 +158,44 @@ function cleanTraceId(value: string | null): string | null {
<header class="isolated-debug-header">
<div class="isolated-debug-title">
<strong>Kafka 卡片隔离调试</strong>
<span>只读独立 debug topic不写产品会话</span>
<span>隔离重放与产品 SSE 原始事件共用一个只读调试卡片</span>
</div>
<div class="isolated-debug-actions">
<span data-testid="workbench-kafka-debug-status"><StatusBadge :status="badgeStatus" :label="statusLabel" /></span>
<button class="btn btn-sm btn-primary" data-testid="workbench-kafka-debug-replay" type="button" :disabled="!traceId || replayBusy" @click="replayCurrentTrace">重放</button>
<button class="btn btn-sm btn-secondary" data-testid="workbench-kafka-debug-clear" type="button" @click="clearPanel">清空隔离面板</button>
<span v-if="activeTab === 'trace'" data-testid="workbench-kafka-debug-status"><StatusBadge :status="badgeStatus" :label="statusLabel" /></span>
<span v-else class="raw-ingress-status" data-testid="workbench-raw-hwlab-status">{{ rawIngress.receivedCount }} frames</span>
<button v-if="activeTab === 'trace' && replayEnabled" class="btn btn-sm btn-primary" data-testid="workbench-kafka-debug-replay" type="button" :disabled="!traceId || replayBusy" @click="replayCurrentTrace">重放</button>
<button v-if="activeTab === 'trace'" class="btn btn-sm btn-secondary" data-testid="workbench-kafka-debug-clear" type="button" @click="clearPanel">清空隔离面板</button>
<button v-else class="btn btn-sm btn-secondary" data-testid="workbench-raw-hwlab-clear" type="button" @click="onClearRawIngress">清空观察窗</button>
</div>
</header>
<div class="isolated-debug-contract" data-testid="workbench-isolated-debug-contract">
<nav class="isolated-debug-tabs" aria-label="Kafka 调试观察模式">
<button v-if="replayEnabled" type="button" data-testid="workbench-kafka-debug-tab-trace" :aria-selected="activeTab === 'trace'" @click="activeTab = 'trace'">Trace 卡片</button>
<button v-if="rawEnabled" type="button" data-testid="workbench-kafka-debug-tab-raw" :aria-selected="activeTab === 'raw'" @click="activeTab = 'raw'">原始 HWLAB Event</button>
</nav>
<div v-if="activeTab === 'trace'" class="isolated-debug-contract" data-testid="workbench-isolated-debug-contract">
<span><b>traceId</b> <code>{{ traceId || "等待当前 turn" }}</code></span>
<span><b>topic</b> <code>{{ connection?.topic || "等待 consumer-ready" }}</code></span>
<span><b>group</b> <code>{{ connection?.groupId || "-" }}</code></span>
<span><b>replayId</b> <code>{{ replayIdLabel }}</code></span>
<span><b>barrier</b> <code>{{ outputBarrierLabel }}</code></span>
<span><b>delivery</b> debug-replay · fromBeginning=true</span>
</div>
<div v-else class="isolated-debug-contract" data-testid="workbench-raw-hwlab-contract">
<span><b>source</b> <code>现有产品 EventSource</code></span>
<span><b>scope</b> <code>{{ rawIngress.scopeKey || "等待产品 SSE scope" }}</code></span>
<span><b>delivery</b> hwlab.event.v1 · memory-only · no client event filter</span>
</div>
<p v-if="visibleError" class="isolated-debug-error">{{ visibleError }}</p>
<p v-if="activeTab === 'trace' && visibleError" class="isolated-debug-error">{{ visibleError }}</p>
<div class="isolated-debug-body">
<div v-if="activeTab === 'trace'" class="isolated-debug-contract" data-testid="workbench-kafka-debug-layer-counts">
<span><b>server</b> scanned={{ serverCounts.scanned ?? "-" }} · matched={{ serverCounts.matched ?? "-" }} · delivered={{ serverCounts.delivered ?? "-" }}</span>
<span><b>client local overlay</b> received={{ clientReceived }} · decoded={{ clientDecoded }} · applied={{ state.appliedCount }}</span>
</div>
<div v-if="activeTab === 'trace'" class="isolated-debug-body">
<div class="isolated-debug-card">
<WorkbenchMessageCard
v-if="state.message"
@@ -153,6 +224,21 @@ function cleanTraceId(value: string | null): string | null {
<div v-else class="isolated-debug-empty compact">尚未收到 debug event</div>
</aside>
</div>
<div v-else class="raw-hwlab-event-window">
<header>
<strong>产品 SSE 原始帧</strong>
<span data-testid="workbench-raw-hwlab-counts">
received={{ rawIngress.receivedCount }} · retained={{ rawIngress.rows.length }} · decoded={{ rawIngress.decodedCount }} · rejected={{ rawIngress.rejectedCount }} · evicted={{ rawIngress.evictedCount }} · bytes={{ rawIngress.retainedBytes }}
</span>
</header>
<textarea
data-testid="workbench-raw-hwlab-text"
readonly
spellcheck="false"
:value="rawText"
:placeholder="rawEnabled ? '等待现有产品 SSE 收到 hwlab.event.v1' : '原始 HWLAB Event capability 已关闭'"
/>
</div>
</section>
</template>
@@ -172,7 +258,8 @@ function cleanTraceId(value: string | null): string | null {
.isolated-debug-header,
.isolated-debug-actions,
.isolated-debug-contract,
.isolated-debug-log header {
.isolated-debug-log header,
.raw-hwlab-event-window header {
display: flex;
align-items: center;
}
@@ -200,6 +287,34 @@ function cleanTraceId(value: string | null): string | null {
gap: 6px;
}
.raw-ingress-status {
color: #6d28d9;
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
font-size: 11px;
}
.isolated-debug-tabs {
display: flex;
gap: 4px;
border-bottom: 1px solid #ddd6fe;
}
.isolated-debug-tabs button {
border: 0;
border-bottom: 2px solid transparent;
background: transparent;
padding: 5px 9px;
color: #6d28d9;
cursor: pointer;
font-size: 12px;
font-weight: 700;
}
.isolated-debug-tabs button[aria-selected="true"] {
border-bottom-color: #7c3aed;
color: #2e1065;
}
.isolated-debug-contract {
min-width: 0;
flex-wrap: wrap;
@@ -302,6 +417,40 @@ function cleanTraceId(value: string | null): string | null {
padding: 7px;
}
.raw-hwlab-event-window {
display: grid;
min-width: 0;
min-height: 0;
grid-template-rows: auto minmax(0, 1fr);
gap: 6px;
}
.raw-hwlab-event-window header {
justify-content: space-between;
gap: 8px;
color: #4c1d95;
font-size: 12px;
}
.raw-hwlab-event-window header span {
color: #6d28d9;
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
font-size: 10px;
}
.raw-hwlab-event-window textarea {
width: 100%;
min-height: 0;
resize: none;
border: 1px solid #ddd6fe;
border-radius: 7px;
background: #111827;
padding: 9px;
color: #e5e7eb;
font: 11px/1.5 ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
white-space: pre;
}
@media (max-width: 900px) {
.isolated-debug-body {
grid-template-columns: minmax(0, 1fr);
@@ -66,3 +66,34 @@ test("mergeRunnerTrace accumulates live Kafka deltas with an honest source and e
assert.equal(merged.eventCount, 2);
assert.deepEqual(merged.events?.map((event) => event.sourceSeq), [1, 2]);
});
test("mergeRunnerTrace deduplicates live Kafka by sourceEventId and retains terminal events", () => {
const previous = trace({
traceId: "trc_live_identity",
eventSource: "hwlab-kafka-sse",
status: "running",
eventCount: 1,
events: [{ sourceEventId: "evt_assistant", runId: "run_live_identity", sourceSeq: 1, type: "assistant", message: "answer" }]
});
const duplicate = trace({
traceId: "trc_live_identity",
eventSource: "hwlab-kafka-sse",
status: "running",
eventCount: 1,
events: [{ sourceEventId: "evt_assistant", runId: "run_live_identity", sourceSeq: 99, type: "assistant", message: "duplicate" }]
});
const deduplicated = mergeRunnerTrace(previous, duplicate);
const terminal = mergeRunnerTrace(deduplicated, trace({
traceId: "trc_live_identity",
eventSource: "hwlab-kafka-sse",
status: "completed",
eventCount: 1,
events: [{ sourceEventId: "evt_terminal", runId: "run_live_identity", sourceSeq: 2, type: "result", terminal: true, status: "completed" }]
}));
assert.equal(deduplicated.eventCount, 1);
assert.equal(deduplicated.events?.[0]?.message, "answer");
assert.equal(terminal.status, "completed");
assert.equal(terminal.eventCount, 2);
assert.deepEqual(terminal.events?.map((event) => event.sourceEventId), ["evt_assistant", "evt_terminal"]);
});
@@ -270,8 +270,12 @@ function traceEventIdentity(event: TraceEvent): string | null {
const seqValue = event.projectedSeq;
const seq = typeof seqValue === "number" || typeof seqValue === "string" ? String(seqValue) : "";
if (seq) return `projected:${seq}:${label || "event"}`;
const source = typeof event.source === "string" ? event.source : "";
const sourceEventId = firstNonEmptyString(event.sourceEventId);
if (sourceEventId) return `source-event:${sourceEventId}`;
const runId = firstNonEmptyString(event.runId);
const sourceSeq = typeof event.sourceSeq === "number" || typeof event.sourceSeq === "string" ? String(event.sourceSeq) : "";
if (runId && sourceSeq) return `run-source:${runId}:${sourceSeq}`;
const source = typeof event.source === "string" ? event.source : "";
if (source && sourceSeq) return `source:${source}:${sourceSeq}:${label || "event"}`;
return null;
}
+23 -3
View File
@@ -17,6 +17,13 @@ export interface WorkbenchRealtimeCapabilities {
export interface WorkbenchDebugCapabilities {
isolatedKafka: boolean;
rawHwlabEventWindow: WorkbenchRawHwlabEventWindowCapability;
}
export interface WorkbenchRawHwlabEventWindowCapability {
enabled: boolean;
maxEntries: number;
maxRetainedBytes: number;
}
const DEFAULT_DISPLAY_DATE_TIME_OPTIONS: Intl.DateTimeFormatOptions = {
@@ -99,10 +106,23 @@ export function workbenchRealtimeCapabilities(): WorkbenchRealtimeCapabilities {
export function workbenchDebugCapabilities(): WorkbenchDebugCapabilities {
const features = window.HWLAB_CLOUD_WEB_CONFIG?.workbench?.debugCapabilities;
if (typeof features?.isolatedKafka !== "boolean") {
throw new Error("HWLAB Cloud Web workbench.debugCapabilities.isolatedKafka is required");
const rawWindow = features?.rawHwlabEventWindow;
if (
typeof features?.isolatedKafka !== "boolean" ||
typeof rawWindow?.enabled !== "boolean" ||
!Number.isInteger(rawWindow.maxEntries) || Number(rawWindow.maxEntries) <= 0 ||
!Number.isInteger(rawWindow.maxRetainedBytes) || Number(rawWindow.maxRetainedBytes) <= 0
) {
throw new Error("HWLAB Cloud Web workbench.debugCapabilities and rawHwlabEventWindow policy are required");
}
return { isolatedKafka: features.isolatedKafka };
return {
isolatedKafka: features.isolatedKafka,
rawHwlabEventWindow: {
enabled: rawWindow.enabled,
maxEntries: Number(rawWindow.maxEntries),
maxRetainedBytes: Number(rawWindow.maxRetainedBytes)
}
};
}
export function traceExplorerHref(traceId: string | null | undefined): string | null {
@@ -0,0 +1,56 @@
import assert from "node:assert/strict";
import { test } from "bun:test";
import type { WorkbenchRawHwlabEventWindowCapability } from "@/config/runtime";
import { appendRawHwlabIngressFrame, createRawHwlabIngressState } from "./workbench-raw-hwlab-ingress";
const capability: WorkbenchRawHwlabEventWindowCapability = {
enabled: true,
maxEntries: 2,
maxRetainedBytes: 32
};
test("raw HWLAB ingress retains complete frames in arrival order and exposes decode rejection", () => {
let state = createRawHwlabIngressState("ses_raw");
const roomyCapability = { ...capability, maxRetainedBytes: 128 };
state = appendRawHwlabIngressFrame(state, frame('{"schema":"hwlab.event.v1"}', "accepted"), roomyCapability, () => 1);
state = appendRawHwlabIngressFrame(state, frame("not-json", "invalid-json"), roomyCapability, () => 2);
assert.equal(state.receivedCount, 2);
assert.equal(state.decodedCount, 1);
assert.equal(state.rejectedCount, 1);
assert.deepEqual(state.rows.map((row) => [row.arrivalSeq, row.rawText, row.decodeStatus]), [
[1, '{"schema":"hwlab.event.v1"}', "accepted"],
[2, "not-json", "invalid-json"]
]);
});
test("raw HWLAB ingress evicts whole envelopes by YAML-owned entry and byte capacities", () => {
let state = createRawHwlabIngressState("ses_raw");
state = appendRawHwlabIngressFrame(state, frame("1234567890", "accepted"), capability, () => 1);
state = appendRawHwlabIngressFrame(state, frame("abcdefghij", "accepted"), capability, () => 2);
state = appendRawHwlabIngressFrame(state, frame("ABCDEFGHIJ", "accepted"), capability, () => 3);
assert.equal(state.receivedCount, 3);
assert.equal(state.evictedCount, 1);
assert.equal(state.retainedBytes, 20);
assert.deepEqual(state.rows.map((row) => row.rawText), ["abcdefghij", "ABCDEFGHIJ"]);
const oversized = appendRawHwlabIngressFrame(state, frame("x".repeat(40), "accepted"), capability, () => 4);
assert.equal(oversized.receivedCount, 4);
assert.equal(oversized.evictedCount, 4);
assert.equal(oversized.retainedBytes, 0);
assert.deepEqual(oversized.rows, []);
});
test("raw HWLAB ingress ignores non-product events and disabled capability", () => {
const state = createRawHwlabIngressState("ses_raw");
const connected = appendRawHwlabIngressFrame(state, { eventName: "workbench.connected", rawText: "{}", decodeStatus: "accepted" }, capability);
const disabled = appendRawHwlabIngressFrame(state, frame("{}", "accepted"), { ...capability, enabled: false });
assert.equal(connected, state);
assert.equal(disabled, state);
});
function frame(rawText: string, decodeStatus: "accepted" | "invalid-json" | "not-object") {
return { eventName: "hwlab.event.v1", rawText, decodeStatus } as const;
}
@@ -0,0 +1,71 @@
// Responsibility: bounded, memory-only observation of the existing product hwlab.event.v1 SSE ingress.
import type { WorkbenchSseIngressFrame } from "@/api/workbench-events";
import type { WorkbenchRawHwlabEventWindowCapability } from "@/config/runtime";
export interface RawHwlabIngressRow {
arrivalSeq: number;
receivedAt: string;
eventName: "hwlab.event.v1";
rawText: string;
decodeStatus: WorkbenchSseIngressFrame["decodeStatus"];
byteLength: number;
}
export interface RawHwlabIngressState {
scopeKey: string;
receivedCount: number;
decodedCount: number;
rejectedCount: number;
evictedCount: number;
retainedBytes: number;
rows: RawHwlabIngressRow[];
}
export function createRawHwlabIngressState(scopeKey = ""): RawHwlabIngressState {
return {
scopeKey,
receivedCount: 0,
decodedCount: 0,
rejectedCount: 0,
evictedCount: 0,
retainedBytes: 0,
rows: []
};
}
export function appendRawHwlabIngressFrame(
state: RawHwlabIngressState,
frame: WorkbenchSseIngressFrame,
capability: WorkbenchRawHwlabEventWindowCapability,
now: () => number = Date.now
): RawHwlabIngressState {
if (!capability.enabled || frame.eventName !== "hwlab.event.v1") return state;
const byteLength = new TextEncoder().encode(frame.rawText).byteLength;
const row: RawHwlabIngressRow = {
arrivalSeq: state.receivedCount + 1,
receivedAt: new Date(now()).toISOString(),
eventName: "hwlab.event.v1",
rawText: frame.rawText,
decodeStatus: frame.decodeStatus,
byteLength
};
const rows = [...state.rows, row];
let retainedBytes = state.retainedBytes + byteLength;
let evictedCount = state.evictedCount;
while (rows.length > capability.maxEntries || retainedBytes > capability.maxRetainedBytes) {
const evicted = rows.shift();
if (!evicted) break;
retainedBytes -= evicted.byteLength;
evictedCount += 1;
}
return {
...state,
receivedCount: state.receivedCount + 1,
decodedCount: state.decodedCount + (frame.decodeStatus === "accepted" ? 1 : 0),
rejectedCount: state.rejectedCount + (frame.decodeStatus === "accepted" ? 0 : 1),
evictedCount,
retainedBytes,
rows
};
}
+15 -3
View File
@@ -4,7 +4,7 @@
import { computed, nextTick, ref } from "vue";
import { defineStore } from "pinia";
import { api } from "@/api";
import { workbenchRealtimeCapabilities } from "@/config/runtime";
import { workbenchDebugCapabilities, workbenchRealtimeCapabilities } from "@/config/runtime";
import { workbenchRuntimePolicy } from "@/config/workbench-runtime-policy";
import { createKeyedSingleflight } from "@/utils/scheduler/keyed-singleflight";
import { createWorkbenchHealthProbeCache } from "@/utils/workbench-health";
@@ -74,6 +74,7 @@ import { useWorkbenchColadaQueries } from "./workbench-colada-queries";
import { useWorkbenchColadaReducer } from "./workbench-colada-reducer";
import { projectionMergeCommitSummary, traceEventsAutoReadDecision, workbenchSessionDetailReadKey, workbenchSessionMessagesReadKey, workbenchTraceEventsReadKey, type TraceEventsReadRangeRecord } from "./workbench-session-messages-read-budget";
import { realtimeSnapshotToTraceSnapshot, terminalSealResultWithoutTraceEvents, traceDetailProjectedSeq, traceNextProjectedSeq } from "./workbench-trace-detail";
import { appendRawHwlabIngressFrame, createRawHwlabIngressState } from "./workbench-raw-hwlab-ingress";
const WORKBENCH_SESSION_PROJECTION_SIGNAL_CHANNEL = "hwlab.workbench.sessionProjection.v1";
const WORKBENCH_SESSION_PROJECTION_SIGNAL_KEY = "hwlab.workbench.sessionProjectionSignal.v1";
@@ -109,6 +110,7 @@ interface RealtimeTurnProjectionItem {
export const useWorkbenchStore = defineStore("workbench", () => {
const runtimePolicy = workbenchRuntimePolicy();
const realtimeCapabilities = workbenchRealtimeCapabilities();
const debugCapabilities = workbenchDebugCapabilities();
const workbenchColadaReducer = useWorkbenchColadaReducer();
const workbenchColadaQueries = useWorkbenchColadaQueries();
const workbenchColadaMutations = useWorkbenchColadaMutations();
@@ -135,6 +137,7 @@ export const useWorkbenchStore = defineStore("workbench", () => {
const sessionListLoadMoreError = ref<string | null>(null);
const chatPending = ref(false);
const liveRealtimeReadySessionId = ref<string | null>(null);
const rawHwlabIngress = ref(createRawHwlabIngressState());
const error = ref<string | null>(null);
const activityRef = ref({ lastActivityAt: Date.now(), lastActivityIso: new Date().toISOString(), waitingFor: "idle", lastEventLabel: null as string | null });
const currentRequest = ref<{ traceId: string; sessionId: string | null; threadId: string | null; status?: string | null } | null>(null);
@@ -1119,8 +1122,10 @@ export const useWorkbenchStore = defineStore("workbench", () => {
const traceId = realtimeTraceId();
const sessionId = selectedSessionId.value ?? null;
void reason;
const scopeChanged = realtimeTransport.currentKey() !== workbenchRealtimeScopeKey(sessionId, traceId);
const realtimeScopeKey = workbenchRealtimeScopeKey(sessionId, traceId);
const scopeChanged = realtimeTransport.currentKey() !== realtimeScopeKey;
if (realtimeCapabilities.liveKafkaSse && scopeChanged) liveRealtimeReadySessionId.value = null;
if (debugCapabilities.rawHwlabEventWindow.enabled && scopeChanged) rawHwlabIngress.value = createRawHwlabIngressState(realtimeScopeKey);
realtimeTransport.restart({
realtimeCapabilities,
sessionId,
@@ -1129,6 +1134,9 @@ export const useWorkbenchStore = defineStore("workbench", () => {
flushMaxItemsPerChunk: runtimePolicy.workbenchRealtimeFlushMaxItemsPerChunk,
flushMaxChunkMs: runtimePolicy.workbenchRealtimeFlushMaxChunkMs,
flushYieldMs: runtimePolicy.workbenchRealtimeFlushYieldMs,
onIngress: debugCapabilities.rawHwlabEventWindow.enabled
? (frame) => { rawHwlabIngress.value = appendRawHwlabIngressFrame(rawHwlabIngress.value, frame, debugCapabilities.rawHwlabEventWindow); }
: undefined,
onOpen: () => undefined,
onError: () => {
if (realtimeCapabilities.liveKafkaSse) liveRealtimeReadySessionId.value = null;
@@ -1943,10 +1951,14 @@ export const useWorkbenchStore = defineStore("workbench", () => {
return typeof performance !== "undefined" && typeof performance.now === "function" ? performance.now() : Date.now();
}
function clearRawHwlabIngress(): void {
rawHwlabIngress.value = createRawHwlabIngressState(rawHwlabIngress.value.scopeKey);
}
installRealtimeVisibilityHandler();
installWorkbenchProjectionSignalHandler();
return { sessions, messages, activeMessages, activeSession, providerProfile, providerOptions, recentDrafts, codeAgentTimeoutMs, gatewayShellTimeoutMs, live, loading, sessionListLoading, sessionDetailLoading, sessionListLoadedCount, sessionListHasMore, sessionListNextCursor, sessionListLoadingMore, sessionListLoadMoreError, chatPending, error, activeSessionId, activeSessionSelectionSource, selectedSessionId, selectedThreadId, sessionTabs, composer, hydrate, refreshLive, createSession, selectSession, selectSessionById, deleteCurrentSession, refreshSessions, loadMoreSessions, submitMessage, cancelAgentMessage, cancelRunningTrace, retryAgentMessage, setProviderProfile, refreshProviderOptions, pickDraft, clearRecentDrafts, clearSessionMessages, recordActivity };
return { sessions, messages, activeMessages, activeSession, providerProfile, providerOptions, recentDrafts, codeAgentTimeoutMs, gatewayShellTimeoutMs, live, loading, sessionListLoading, sessionDetailLoading, sessionListLoadedCount, sessionListHasMore, sessionListNextCursor, sessionListLoadingMore, sessionListLoadMoreError, chatPending, error, rawHwlabIngress, activeSessionId, activeSessionSelectionSource, selectedSessionId, selectedThreadId, sessionTabs, composer, hydrate, refreshLive, createSession, selectSession, selectSessionById, deleteCurrentSession, refreshSessions, loadMoreSessions, submitMessage, cancelAgentMessage, cancelRunningTrace, retryAgentMessage, setProviderProfile, refreshProviderOptions, pickDraft, clearRecentDrafts, clearSessionMessages, clearRawHwlabIngress, recordActivity };
});
function workbenchSessionsFromPayload(payload: unknown): WorkbenchSessionRecord[] {
+5
View File
@@ -14,6 +14,11 @@ declare global {
};
debugCapabilities?: {
isolatedKafka?: boolean;
rawHwlabEventWindow?: {
enabled?: boolean;
maxEntries?: number;
maxRetainedBytes?: number;
};
};
traceTimeline?: {
autoExpandRunning?: boolean;
@@ -7,7 +7,7 @@
// - OpenCode stream.transport.ts:391-443 stream acquire/subscribe.
// - OpenCode stream.transport.ts:504-530 reduce output/trace.
import { connectWorkbenchEvents, type WorkbenchEventStream, type WorkbenchRealtimeEvent } from "@/api/workbench-events";
import { connectWorkbenchEvents, type WorkbenchEventStream, type WorkbenchRealtimeEvent, type WorkbenchSseIngressFrame } from "@/api/workbench-events";
import type { WorkbenchRealtimeCapabilities } from "@/config/runtime";
import { workbenchRealtimeScopeKey } from "@/utils/workbench-key";
@@ -22,6 +22,7 @@ export interface WorkbenchStreamTransportRestartInput {
flushMaxItemsPerChunk?: number | null;
flushMaxChunkMs?: number | null;
flushYieldMs?: number | null;
onIngress?: (frame: WorkbenchSseIngressFrame) => void;
onOpen?: () => void;
onError?: (event: Event) => void;
onState?: (state: WorkbenchStreamTransportState) => void;
@@ -117,6 +118,7 @@ export class WorkbenchStreamTransportRuntime {
flushMaxItemsPerChunk: input.flushMaxItemsPerChunk,
flushMaxChunkMs: input.flushMaxChunkMs,
flushYieldMs: input.flushYieldMs,
onIngress: input.onIngress,
onOpen: () => {
this.markLive(key);
this.emitState(input, "open", null);
@@ -30,7 +30,7 @@ const routeReflectionTarget = computed(() => routeRequestId.value ?? (rawRouteSe
const applyingRouteSession = ref(Boolean(rawRouteSessionId.value));
const componentActive = ref(false);
const isolatedDebugOpen = ref(false);
const isolatedDebugAvailable = computed(() => debugCapabilities.isolatedKafka && auth.isAdmin);
const isolatedDebugAvailable = computed(() => auth.isAdmin && (debugCapabilities.isolatedKafka || debugCapabilities.rawHwlabEventWindow.enabled));
const isolatedDebugTraceId = computed(() => workbenchCurrentDebugTraceId(workbench.activeMessages, workbench.activeSession?.lastTraceId));
type LaunchContextRecord = {
sourceId?: string | null;
@@ -135,12 +135,19 @@ async function reflectActiveSessionInUrl(value: string | null): Promise<void> {
<div v-if="isolatedDebugAvailable" class="workbench-isolated-debug-toggle">
<label>
<input v-model="isolatedDebugOpen" data-testid="workbench-kafka-debug-toggle" type="checkbox" />
<span>隔离 Kafka 调试模式</span>
<span>Kafka 调试观察模式</span>
</label>
<small>{{ isolatedDebugTraceId ? `当前 trace ${isolatedDebugTraceId}` : "等待当前 traceId" }}</small>
</div>
<ConversationPanel />
<WorkbenchKafkaDebugPanel v-if="isolatedDebugOpen" :trace-id="isolatedDebugTraceId" />
<WorkbenchKafkaDebugPanel
v-if="isolatedDebugOpen"
:trace-id="isolatedDebugTraceId"
:replay-enabled="debugCapabilities.isolatedKafka"
:raw-enabled="debugCapabilities.rawHwlabEventWindow.enabled"
:raw-ingress="workbench.rawHwlabIngress"
:on-clear-raw-ingress="workbench.clearRawHwlabIngress"
/>
</div>
<CommandComposer />
</main>