mirrors/pikasTech-HWLAB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,280 Commits 3 Branches 0 Tags
d4929bcbb11dbbddeb940d88f3cd0cf3b71f9d8b
Commit Graph

488 Commits

Author SHA1 Message Date
lyon ea0354aca2 instrument workbench runtime otel spans 2026-06-21 21:55:52 +08:00
lyon 2bafed5b85 fix workbench runtime nullable json reads 2026-06-21 21:38:30 +08:00
lyon 0fe111747e move workbench session list to go runtime 2026-06-21 21:25:14 +08:00
lyon cf9ac0b08e feat: split workbench runtime read service 2026-06-21 19:42:26 +08:00
Lyon e23e9522d1 fix: keep workbench alive on postgres timeout (#1831) 2026-06-21 19:20:34 +08:00
Lyon 0833e115bf fix: reduce workbench read model load (#1829) 2026-06-21 18:41:07 +08:00
Lyon 507850579e fix: stabilize workbench turn diagnostics (#1828) 2026-06-21 18:23:13 +08:00
lyon 84180e3ae2 fix: do not seal running agentrun turns on caught-up projection 2026-06-21 14:47:06 +08:00
Lyon a5b47d5d8d fix: expose workbench projection failures in otel (#1824) 2026-06-21 13:44:22 +08:00
Lyon c3abe1a51a fix: seal AgentRun terminal projection (#1820) 2026-06-21 13:17:37 +08:00
Lyon 70d90e85eb fix: emit workbench sse accepted span (#1816) 2026-06-21 12:20:25 +08:00
Lyon 0f3bc3f99a fix workbench otel http span timing (#1815) 2026-06-21 12:07:37 +08:00
Lyon 6cafd495de fix(cloud-api): keep health live nonblocking (#1808) 
Route /health/live through cached runtime summary so DB durable retry cannot block liveness probes. Keep /health as the full durable readiness endpoint.\n\nRefs #1807.
 2026-06-21 10:28:02 +08:00
Lyon a0c54934b2 fix(workbench): lengthen projection resume retry delay (#1804) 2026-06-21 09:57:26 +08:00
Lyon 0c25d44eac fix(workbench): back off non-terminal projection resumes (#1803) 2026-06-21 09:51:15 +08:00
Lyon bc8805baaf fix(workbench): back off projection resume pass failures (#1802) 2026-06-21 09:45:11 +08:00
Lyon 5b66e9db10 fix(workbench): back off projection resume candidates (#1801) 2026-06-21 09:39:10 +08:00
lyon 6e05c05732 fix(workbench): seal message status from turn projection 2026-06-21 07:52:59 +08:00
lyon b6dc2b4126 fix(workbench): seal final response lifecycle 2026-06-21 07:26:22 +08:00
lyon eedd8b6bfb fix(workbench): keep provider retry diagnostics non-terminal 2026-06-21 07:12:33 +08:00
lyon e9d19c8617 fix: retry transient workbench facts reads 2026-06-21 05:08:08 +08:00
lyon 7ac200cd7e fix: expose retryable projection gaps 2026-06-21 04:14:58 +08:00
lyon adba4a4354 fix: contain postgres unhandled connection timeouts 2026-06-21 04:00:50 +08:00
lyon cf13bf80a9 fix: contain postgres projection timeouts 2026-06-21 03:45:23 +08:00
lyon 1f59fbd33e fix: keep running turn timing authoritative 2026-06-21 03:29:29 +08:00
lyon 0d861d03ec fix: prioritize terminal checkpoint status 2026-06-21 03:21:09 +08:00
lyon e30e9e84aa fix: keep turn timing independent from sessions 2026-06-21 03:08:14 +08:00
lyon bcc8aa189b fix: stabilize workbench timing diagnostics 2026-06-21 02:56:19 +08:00
lyon a6b5e9a975 fix: harden workbench agent retry projection 2026-06-21 02:43:33 +08:00
lyon 49e42be440 fix(workbench): derive session terminal status from durable trace 2026-06-21 01:56:09 +08:00
Lyon cd981355c6 fix(workbench): keep per-turn message timing isolated (#1768) 2026-06-21 00:56:12 +08:00
Lyon fb1fe8d94a feat(otel): 服务端 HTTP request span 追穿 hwlab-cloud-api 503 (#1764) 
问题：hwlab-cloud-api 服务端 503（如 workbench 投影存储不可用）在 Tempo
里只有 workbench-ui 浏览器侧 span，没有服务端 request handler span，导致
后端根因（error.code=projection_store_unavailable 等）只能在应用日志里看，
OTel 链路追不到服务端处理层。

修复：
- otel-trace.ts 新增 emitHttpServerRequestSpan（scope hwlab.http.server，
  SPAN_KIND_SERVER），捕获 http.route/method/status_code/status_class、
  request.id，5xx 时落 STATUS_CODE_ERROR 并把 error.code/category/layer
  写入 span 属性与 status.message。
- error-envelope.ts 的 decorateErrorPayloadForHttp 把诊断 error code/
  category/layer 回写到 hwlabHttpRequestContext.lastHttpError，供 finish
  时复用，避免重复解析。
- server.ts 在 response finish 时按 http context + backend performance
  时序发服务端 span；health/metrics 探针路由跳过，避免 OTel 噪声洪泛。

验收：canary 调用 emitHttpServerRequestSpan 发 503 projection_store_unavailable
span 到 D601 platform-infra OTel collector，Tempo 可查到 service=hwlab-cloud-api、
scope=hwlab.http.server、http.response.status_code=503、error.code=
projection_store_unavailable 的 ERROR span。
 2026-06-20 20:31:36 +08:00
lyon bad5a478bd fix: recover Workbench terminal outbox projection 2026-06-20 20:12:01 +08:00
lyon 800f674d5c fix: recover AgentRun terminal projection 2026-06-20 20:05:13 +08:00
lyon 6ab94f23c8 fix: clarify workbench trace event diagnostics 2026-06-20 19:43:54 +08:00
Lyon 4f7e55e2d8 fix: PJ2026-0104010803 P5 残余 events.at(-1) 读侧推理清理 + code-agent metric status (#1759) 
useTraceSubscription.ts:
- mergeTraceResults: lastEventLabel 不 fallback 到 events.at(-1)
- mergeTraceSnapshots: lastEventLabel 不 fallback 到 events.at(-1)
- traceSnapshotSignalKey: 不从 events.at(-1) 推导 signal key

server-code-agent-http.ts:
- recordCodeAgentProjectionMetric: status 只读 projection.status
- source 固定为 workbench_read_model

负向源码扫描确认:
- result?.status ?? trace?.status: 消除
- events.at(-1) lifecycle 推断: 消除
- .catch(() => {}) 静默吞错: 消除
- traceStore.subscribe: 仅作为 outbox 不可用时的兼容降级

Closes #1751
Refs #1742
 2026-06-20 19:42:19 +08:00
Lyon 9574e2b514 feat: PJ2026-0104010803 P3 后端读侧推理清零 - session list + metric status 只读 projection (#1757) 
server-workbench-http.ts:
- sessionListProjectionOptions: 当 session snapshot 已有 durable projection 字段
  (projectionStatus/terminal/sealedAt) 时，直接使用 projection 字段构建 turn projection，
  不再从 compactSessionTraceResult/compactSessionTraceSnapshot 多来源混合
- recordWorkbenchProjectionMetric: status 只读 projection.status，不再 fallback 到
  result?.status ?? trace?.status；source 固定为 workbench_read_model
- SPEC header 更新到 draft-2026-06-20-p3-read-side-zero

workbench-turn-projection.ts:
- SPEC header 更新到 draft-2026-06-20-p1-zero-split-durable-realtime

Closes #1749
Refs #1742
 2026-06-20 19:34:52 +08:00
Lyon 7c5a1f8c5b feat: PJ2026-0104010803 P2 SSE/realtime 无状态化 - durable outbox cursor replay (#1756) 
server-workbench-http.ts:
- handleWorkbenchRealtimeHttp 优先从 durable outbox 读取 (readWorkbenchProjectionOutbox)
- 当 runtimeStore 支持 outbox 时，SSE 用 polling cursor replay 代替 traceStore.subscribe
- outbox 不支持时保留内存 traceStore.subscribe 作为兼容降级
- terminal commit 从 outbox terminal 行触发 snapshot + trace realtime
- 新增 HWLAB_WORKBENCH_SSE_OUTBOX_POLL_MS 环境变量控制 poll 间隔

workbench-facts-store.ts:
- 新增 subscribeDurableProjection 方法，从 durable outbox polling 读取
- subscribeTrace 保留兼容（内存 trace），新增 durable projection subscription 入口
- SPEC header 更新到 draft-2026-06-20-p1-zero-split-durable-realtime

cloud-api 重启后 SSE 从 DB outbox cursor 继续输出，不依赖内存 listener

Closes #1748
Refs #1742
 2026-06-20 19:31:49 +08:00
Lyon b254c1875e feat: PJ2026-0104010803 P1 durable projection outbox + DB constraints + error visibility (#1754) 
DB migration (0001_cloud_core_skeleton.sql):
- Add (trace_id, source_event_id) unique index on workbench_trace_events
- Add (trace_id, projected_seq) unique index on workbench_trace_events
- Add workbench_projection_outbox table with BIGSERIAL outbox_seq

Schema (schema.ts):
- Bump CLOUD_RUNTIME_DURABLE_ADAPTER_SCHEMA_VERSION to v5
- Add workbench_projection_outbox to CLOUD_RUNTIME_DURABLE_TABLE_COLUMNS

Runtime store (runtime-store.ts):
- Add persistWorkbenchProjectionOutbox method
- Add readWorkbenchProjectionOutbox method for SSE cursor replay
- Write outbox rows in writeWorkbenchFacts for trace events and terminal turns
- Outbox write errors are logged, not silently swallowed

Trace store (code-agent-trace-store.ts):
- persistTraceEvent no longer uses .catch(() => {}) to swallow errors
- Projection write errors are logged with traceId/sourceEventId for diagnostics
- Sync errors also logged instead of empty catch block

SPEC headers updated to draft-2026-06-20-p1-zero-split-durable-realtime

Closes #1747
Refs #1742
 2026-06-20 19:29:44 +08:00
lyon 062df3bf49 test: align user billing async agentrun coverage 2026-06-20 18:17:30 +08:00
lyon 3718b5d77d fix: persist manual workbench sessions 2026-06-20 16:58:31 +08:00
lyon 11e2f08e70 fix: order workbench sessions by updated desc 2026-06-20 15:36:41 +08:00
lyon b139ffd8e4 feat: normalize cloud api error diagnostics 2026-06-20 15:08:33 +08:00
Lyon 199ee18b58 Merge pull request #1718 from pikasTech/feat/1708-p1-trace-context 
[1708-P1] 贯通 Cloud API OTel trace 响应头
 2026-06-20 14:55:53 +08:00
lyon efafeccfb9 feat: attach cloud api otel trace headers 2026-06-20 14:54:51 +08:00
lyon 03942afc5c fix: 收敛 workbench 消息头 timing 投影 2026-06-20 14:54:04 +08:00
lyon 4a9498d720 fix(workbench): expose code agent timing metadata 2026-06-20 13:55:57 +08:00
Lyon d2e147c019 fix: align performance summary problem counts (#1703) 2026-06-20 13:49:10 +08:00
lyon 6468263672 fix: thin workbench session list projection 2026-06-20 13:24:49 +08:00
Lyon 96704ea1d7 Merge pull request #1695 from pikasTech/codex/hwlab-1691-problem-rows 
fix: separate performance problems from sample health
 2026-06-20 12:47:57 +08:00