Commit Graph

469 Commits

Author SHA1 Message Date
lyon eedd8b6bfb fix(workbench): keep provider retry diagnostics non-terminal 2026-06-21 07:12:33 +08:00
lyon e9d19c8617 fix: retry transient workbench facts reads 2026-06-21 05:08:08 +08:00
lyon 7ac200cd7e fix: expose retryable projection gaps 2026-06-21 04:14:58 +08:00
lyon adba4a4354 fix: contain postgres unhandled connection timeouts 2026-06-21 04:00:50 +08:00
lyon cf13bf80a9 fix: contain postgres projection timeouts 2026-06-21 03:45:23 +08:00
lyon 1f59fbd33e fix: keep running turn timing authoritative 2026-06-21 03:29:29 +08:00
lyon 0d861d03ec fix: prioritize terminal checkpoint status 2026-06-21 03:21:09 +08:00
lyon e30e9e84aa fix: keep turn timing independent from sessions 2026-06-21 03:08:14 +08:00
lyon bcc8aa189b fix: stabilize workbench timing diagnostics 2026-06-21 02:56:19 +08:00
lyon a6b5e9a975 fix: harden workbench agent retry projection 2026-06-21 02:43:33 +08:00
lyon 49e42be440 fix(workbench): derive session terminal status from durable trace 2026-06-21 01:56:09 +08:00
Lyon cd981355c6 fix(workbench): keep per-turn message timing isolated (#1768) 2026-06-21 00:56:12 +08:00
Lyon fb1fe8d94a feat(otel): 服务端 HTTP request span 追穿 hwlab-cloud-api 503 (#1764)
问题:hwlab-cloud-api 服务端 503(如 workbench 投影存储不可用)在 Tempo
里只有 workbench-ui 浏览器侧 span,没有服务端 request handler span,导致
后端根因(error.code=projection_store_unavailable 等)只能在应用日志里看,
OTel 链路追不到服务端处理层。

修复:
- otel-trace.ts 新增 emitHttpServerRequestSpan(scope hwlab.http.server,
  SPAN_KIND_SERVER),捕获 http.route/method/status_code/status_class、
  request.id,5xx 时落 STATUS_CODE_ERROR 并把 error.code/category/layer
  写入 span 属性与 status.message。
- error-envelope.ts 的 decorateErrorPayloadForHttp 把诊断 error code/
  category/layer 回写到 hwlabHttpRequestContext.lastHttpError,供 finish
  时复用,避免重复解析。
- server.ts 在 response finish 时按 http context + backend performance
  时序发服务端 span;health/metrics 探针路由跳过,避免 OTel 噪声洪泛。

验收:canary 调用 emitHttpServerRequestSpan 发 503 projection_store_unavailable
span 到 D601 platform-infra OTel collector,Tempo 可查到 service=hwlab-cloud-api、
scope=hwlab.http.server、http.response.status_code=503、error.code=
projection_store_unavailable 的 ERROR span。
2026-06-20 20:31:36 +08:00
lyon bad5a478bd fix: recover Workbench terminal outbox projection 2026-06-20 20:12:01 +08:00
lyon 800f674d5c fix: recover AgentRun terminal projection 2026-06-20 20:05:13 +08:00
lyon 6ab94f23c8 fix: clarify workbench trace event diagnostics 2026-06-20 19:43:54 +08:00
Lyon 4f7e55e2d8 fix: PJ2026-0104010803 P5 残余 events.at(-1) 读侧推理清理 + code-agent metric status (#1759)
useTraceSubscription.ts:
- mergeTraceResults: lastEventLabel 不 fallback 到 events.at(-1)
- mergeTraceSnapshots: lastEventLabel 不 fallback 到 events.at(-1)
- traceSnapshotSignalKey: 不从 events.at(-1) 推导 signal key

server-code-agent-http.ts:
- recordCodeAgentProjectionMetric: status 只读 projection.status
- source 固定为 workbench_read_model

负向源码扫描确认:
- result?.status ?? trace?.status: 消除
- events.at(-1) lifecycle 推断: 消除
- .catch(() => {}) 静默吞错: 消除
- traceStore.subscribe: 仅作为 outbox 不可用时的兼容降级

Closes #1751
Refs #1742
2026-06-20 19:42:19 +08:00
Lyon 9574e2b514 feat: PJ2026-0104010803 P3 后端读侧推理清零 - session list + metric status 只读 projection (#1757)
server-workbench-http.ts:
- sessionListProjectionOptions: 当 session snapshot 已有 durable projection 字段
  (projectionStatus/terminal/sealedAt) 时,直接使用 projection 字段构建 turn projection,
  不再从 compactSessionTraceResult/compactSessionTraceSnapshot 多来源混合
- recordWorkbenchProjectionMetric: status 只读 projection.status,不再 fallback 到
  result?.status ?? trace?.status;source 固定为 workbench_read_model
- SPEC header 更新到 draft-2026-06-20-p3-read-side-zero

workbench-turn-projection.ts:
- SPEC header 更新到 draft-2026-06-20-p1-zero-split-durable-realtime

Closes #1749
Refs #1742
2026-06-20 19:34:52 +08:00
Lyon 7c5a1f8c5b feat: PJ2026-0104010803 P2 SSE/realtime 无状态化 - durable outbox cursor replay (#1756)
server-workbench-http.ts:
- handleWorkbenchRealtimeHttp 优先从 durable outbox 读取 (readWorkbenchProjectionOutbox)
- 当 runtimeStore 支持 outbox 时,SSE 用 polling cursor replay 代替 traceStore.subscribe
- outbox 不支持时保留内存 traceStore.subscribe 作为兼容降级
- terminal commit 从 outbox terminal 行触发 snapshot + trace realtime
- 新增 HWLAB_WORKBENCH_SSE_OUTBOX_POLL_MS 环境变量控制 poll 间隔

workbench-facts-store.ts:
- 新增 subscribeDurableProjection 方法,从 durable outbox polling 读取
- subscribeTrace 保留兼容(内存 trace),新增 durable projection subscription 入口
- SPEC header 更新到 draft-2026-06-20-p1-zero-split-durable-realtime

cloud-api 重启后 SSE 从 DB outbox cursor 继续输出,不依赖内存 listener

Closes #1748
Refs #1742
2026-06-20 19:31:49 +08:00
Lyon b254c1875e feat: PJ2026-0104010803 P1 durable projection outbox + DB constraints + error visibility (#1754)
DB migration (0001_cloud_core_skeleton.sql):
- Add (trace_id, source_event_id) unique index on workbench_trace_events
- Add (trace_id, projected_seq) unique index on workbench_trace_events
- Add workbench_projection_outbox table with BIGSERIAL outbox_seq

Schema (schema.ts):
- Bump CLOUD_RUNTIME_DURABLE_ADAPTER_SCHEMA_VERSION to v5
- Add workbench_projection_outbox to CLOUD_RUNTIME_DURABLE_TABLE_COLUMNS

Runtime store (runtime-store.ts):
- Add persistWorkbenchProjectionOutbox method
- Add readWorkbenchProjectionOutbox method for SSE cursor replay
- Write outbox rows in writeWorkbenchFacts for trace events and terminal turns
- Outbox write errors are logged, not silently swallowed

Trace store (code-agent-trace-store.ts):
- persistTraceEvent no longer uses .catch(() => {}) to swallow errors
- Projection write errors are logged with traceId/sourceEventId for diagnostics
- Sync errors also logged instead of empty catch block

SPEC headers updated to draft-2026-06-20-p1-zero-split-durable-realtime

Closes #1747
Refs #1742
2026-06-20 19:29:44 +08:00
lyon 062df3bf49 test: align user billing async agentrun coverage 2026-06-20 18:17:30 +08:00
lyon 3718b5d77d fix: persist manual workbench sessions 2026-06-20 16:58:31 +08:00
lyon 11e2f08e70 fix: order workbench sessions by updated desc 2026-06-20 15:36:41 +08:00
lyon b139ffd8e4 feat: normalize cloud api error diagnostics 2026-06-20 15:08:33 +08:00
Lyon 199ee18b58 Merge pull request #1718 from pikasTech/feat/1708-p1-trace-context
[1708-P1] 贯通 Cloud API OTel trace 响应头
2026-06-20 14:55:53 +08:00
lyon efafeccfb9 feat: attach cloud api otel trace headers 2026-06-20 14:54:51 +08:00
lyon 03942afc5c fix: 收敛 workbench 消息头 timing 投影 2026-06-20 14:54:04 +08:00
lyon 4a9498d720 fix(workbench): expose code agent timing metadata 2026-06-20 13:55:57 +08:00
Lyon d2e147c019 fix: align performance summary problem counts (#1703) 2026-06-20 13:49:10 +08:00
lyon 6468263672 fix: thin workbench session list projection 2026-06-20 13:24:49 +08:00
Lyon 96704ea1d7 Merge pull request #1695 from pikasTech/codex/hwlab-1691-problem-rows
fix: separate performance problems from sample health
2026-06-20 12:47:57 +08:00
Lyon acd9831149 Fix workbench trace projection authority (#1696) 2026-06-20 12:47:18 +08:00
lyon 7448803c9d fix: separate performance problems from sample health 2026-06-20 12:46:29 +08:00
lyon 8c27242656 fix: scope workbench performance attribution health 2026-06-20 12:32:44 +08:00
lyon c8ff65f11f fix: reuse proven runtime readiness for workbench reads 2026-06-20 12:17:54 +08:00
lyon c9028519ca fix: overlap workbench session include query 2026-06-20 12:01:30 +08:00
Lyon ecbb13cdf4 fix: expose trace pagination in cli and otel (#1687) 2026-06-20 11:32:13 +08:00
lyon 1c4cf29274 feat: enrich AgentRun dispatch error OTel spans 2026-06-20 11:17:57 +08:00
lyon 793f0f39a4 fix: use projected trace cursor as workbench authority 2026-06-20 10:55:11 +08:00
lyon 1e484005c3 fix: reset invalid AgentRun profile session before run create 2026-06-20 10:40:32 +08:00
Lyon f637f32631 fix: parallelize workbench read-model fact reads (#1680) 2026-06-20 10:29:00 +08:00
lyon c3de9d6e4d fix: allow provider profile recovery after thread resume failure 2026-06-20 10:26:50 +08:00
lyon 28a7eaadfa fix: ignore poisoned AgentRun session thread after resume failure 2026-06-20 10:19:36 +08:00
lyon e91f9b5c0e fix: scope AgentRun thread reuse to backend profile session 2026-06-20 10:11:29 +08:00
Lyon 57ba729fca fix: reduce workbench read-model fact scans (#1676) 2026-06-20 10:02:38 +08:00
lyon c51dd122e2 fix: 区分 provider retry 与终态失败 2026-06-20 09:51:30 +08:00
Lyon 40a54eef20 feat: align rum samples with backend evidence (#1674) 2026-06-20 09:33:54 +08:00
lyon 4e64058614 feat: read workbench get from durable facts 2026-06-20 09:15:59 +08:00
Lyon e0127f1044 feat: expose performance collection health (#1671) 2026-06-20 09:03:17 +08:00
lyon e79ea2cb42 feat: persist workbench projection facts 2026-06-20 08:57:21 +08:00