fix(web): v0.2 remove all total-timeout / hard cap on Code Agent turn (HWLAB #795 final) (#798)

Follow-up to PR #797: drop every wall-clock or attempt-count ceiling from
`state/runner-trace.ts` so the Web UI's Code Agent inactivity-timeout
contract is total-cap-free. The previous PR kept a
`max(totalTimeoutMs * 4, totalTimeoutMs + 60s)` outer-loop safety net
plus a `TRACE_HARD_CAP_ATTEMPTS = 120` poll-count cap, which would
re-introduce the same regression class that #795 was filed against: a
long-running operation that keeps emitting trace events could still be
killed by the outer ceiling even with an active `activityRef`. The
"Code Agent Timeout Model" section of
`docs/reference/spec-v02-hwlab-cloud-web.md` now pins the contract:

- The sole abort signal is per-poll `fetchJson` inactivity-timeout,
  driven by `activityRef`.
- The outer `waitForAgentResult` / `pollRunnerTrace` loops are
  `for (;;)`. They only exit when the upstream returns a terminal
  status, the request fails with a non-5xx, or the inactivity window
  fires inside `fetchJson`.
- The per-poll `getAgentChatResult` call passes the full
  `totalTimeoutMs`; the inactivity window is **not** shrunk by
  wall-clock elapsed time.
- `TRACE_HARD_CAP_ATTEMPTS` and the 4x / +60s outer cap are deleted.
- Runaway protection is the caller's responsibility: Web users have
  cancel / steer / close-tab; CLI has `--timeout-ms`. The browser does
  not introduce an implicit cap.

Changes

- `web/hwlab-cloud-web/src/state/runner-trace.ts`:
  - Drop `const hardCapMs = Math.max(totalTimeoutMs * 4, totalTimeoutMs + 60_000)`
    and the `if (Date.now() - startedAt >= hardCapMs) break;` checks in
    both `waitForAgentResult` and `pollRunnerTrace`.
  - Replace `while (Date.now() - startedAt < hardCapMs)` with
    `for (;;)`. Drop the now-unused `startedAt` locals and the
    `let attempt = 0; attempt > TRACE_HARD_CAP_ATTEMPTS` guard.
  - Pass `totalTimeoutMs` directly to `api.getAgentChatResult` instead
    of `totalTimeoutMs - (Date.now() - startedAt)` so the per-poll
    inactivity window is stable.
  - Delete `const TRACE_HARD_CAP_ATTEMPTS = 120;` (no longer referenced).

- `web/hwlab-cloud-web/scripts/fetchJson-inactivity.test.ts`:
  - Replace the "1.5s cadence 6.5s" case with a stricter "200ms cadence
    10s" case: `timeoutMs=3000` inactivity window, continuous 200ms
    activity for 10s, then stop. Assert `elapsed >= 10_000` and that
    the abort is the inactivity-timeout error. This pins the
    `elapsed >= 10000` invariant in the spec.

- `docs/reference/spec-v02-hwlab-cloud-web.md`:
  - Strengthen the existing bullet in "在系统中的职责划分" to
    explicitly forbid total-timeout, hard cap, `codeAgentTimeoutMs * N`,
    poll-count cap, and wall-clock-shrunk per-poll windows.
  - Add a dedicated "Code Agent Timeout Model" section after "## 内部架构"
    that distils the contract, lists the three invariants, and records
    the history of #775 / #777 / #791 / #795 leading to the final state.

Verification

- `bun run scripts/tsc-check.ts` → strict React TSX, 0 explicit any
- `bun run check` → 5 pass / 0 fail (3 inactivity + 2 dist-contract)
- `bun run build` → 9 dist files verified fresh; `app.js` 241802 → 238460 B
  (delta reflects the deleted hardCap / TRACE_HARD_CAP_ATTEMPTS code)
- New 200ms-cadence 10s inactivity test runs for ~13s and asserts the
  request lives the full window — proving the outer cap is gone.

Refs: HWLAB #795 (final), #777, #791, #775

Co-authored-by: Codex <codex@local>
This commit is contained in:
Lyon
2026-06-04 09:36:29 +08:00
committed by GitHub
parent 909715c335
commit f7731d1421
3 changed files with 58 additions and 29 deletions
+24 -1
View File
@@ -21,11 +21,34 @@
- runner pod 被删、runner Job 重建或旧 lease 失效后的临时恢复,可以显示新的 AgentRun run/job/command identity,但 Web 必须继续以同一个 HWLAB `sessionId` / `threadId` / provider profile 呈现业务会话,并明确区分“同 session 恢复执行壳”和“新业务 session”。只要复用同一 AgentRun `SessionRef`/PVC/thread 且没有拼接历史 promptreplacement run/job 可以作为 session 持久化恢复证据;它不替代 T2.2 的同 run/runner reuse 目标。
- AgentRun 会话连续性只有一个标准路径:Cloud Web/CLI 提交的 `threadId` 必须经 Cloud API adapter 写入 AgentRun command `payload.threadId``SessionRef.threadId`。前端、CLI、API 和 AgentRun 的协议字段、trace、result 和 conversation facts 都以该字段为唯一 thread identity。
- Cloud Web 提交 Code Agent turn 时只发送当前用户消息、共享 workspace 的 `conversationId/sessionId/threadId`、workspace revision 和必要运行元数据;不得发送 `conversationContext/messages`,也不得把浏览器历史拼入 prompt。历史消息只用于本地 UI 展示和 trace/inspect 可见性,不能替代 AgentRun/Codex stdio 原生 `thread/resume`
- Workbench 不允许把 Code Agent 长任务总耗时当成失败条件;只能在无新 trace/event/activity 的 idle timeout 后显示超时。AgentRun command/provider/backend 失败后,当前消息可显示 failed/blockersession 是否仍 usable 必须由 session 状态显式表达。`thread-resume-failed`、provider continuation 失效、运行面中断或用户取消导致 session failed/stale/canceled 时,不得自动清理并滚动到新 session;继续前必须由用户显式创建或选择 session。
- **Code Agent Timeout ModelHWLAB #795 final**Workbench 不允许把 Code Agent 长任务总耗时当成失败条件,也不得在浏览器侧引入任何 total-timeout / hard cap / `codeAgentTimeoutMs * N` 兜底 / 轮询次数上限 / 按 wall-clock 缩小的 per-poll 窗口。Code Agent turn 的唯一 abort 信号是 **inactivity-timeout**:即 `fetchJson``timeoutMs`(用户配的 `codeAgentTimeoutMs`)窗口内没有任何新 activitytrace snapshot / user typing / submit kickoff / server 5xx 重试)。一旦 `activityRef.lastActivityAt` 在窗口内被刷新,per-poll 窗口必须维持原值不缩小,外层 `waitForAgentResult` / `pollRunnerTrace``for(;;)` 循环也没有任何累计计时跳出条件。Runaway 保护的责任在调用方:用户可以走 cancel / steer / 关 tab / 关浏览器 / session cancel,而不是浏览器隐式地按 `4 × totalTimeoutMs` 把活跃 turn 杀掉。AgentRun command/provider/backend 失败后,当前消息可显示 failed/blockersession 是否仍 usable 必须由 session 状态显式表达。`thread-resume-failed`、provider continuation 失效、运行面中断或用户取消导致 session failed/stale/canceled 时,不得自动清理并滚动到新 session;继续前必须由用户显式创建或选择 session。
- 同一 AgentRun run 复用多条 command 时,Web trace 展示只显示当前 command 归属事件和必要 run 级状态;旧 command 的 assistant/tool/terminal 尾部不能堆到新 command 末尾。取消轮次的可读进展必须作为脱敏 conversation facts 进入 UI/trace/inspect 证据,而不是靠旧 trace 尾部串线让后续轮次“碰巧看到”;这些 facts 不得作为下一轮模型上下文或 prompt 拼接来源。
## 内部架构
## Code Agent Timeout Model(与 #795 PR 同步固化)
权威规则:Cloud Web 的 `state/workbench.ts` `submitMessage``state/runner-trace.ts` `waitForAgentResult` / `pollRunnerTrace` 必须遵守以下 contract,禁止用任何 wall-clock / 轮询次数 / 4× / +60s 兜底来"安全网"式杀掉活跃 turn
- **唯一 abort 信号**per-poll `fetchJson` 的 inactivity-timeout,由 `activityRef` 驱动。`activityRef.lastActivityAt``totalTimeoutMs` 窗口内被刷新就不 abort。
- **没有 total-timeout**`waitForAgentResult``pollRunnerTrace` 的外层循环是 `for (;;)`,没有 `Date.now() - startedAt >= <cap>` 的跳出。per-poll `getAgentChatResult(url, totalTimeoutMs, activityRef)` 直接传 `totalTimeoutMs`**不**随墙钟缩小成 `totalTimeoutMs - (Date.now() - startedAt)`
- **没有轮询次数上限**`TRACE_HARD_CAP_ATTEMPTS` 已删除。"轮询 120 次还没结束"不是 abort 条件;如果上游永远不返回 terminal 且永远不说话,浏览器就持续 poll 到用户 cancel / 关 tab。
- **没有 `Math.max(totalTimeoutMs * N, ...)` 兜底**:之前 round 9 留的 `max(4x, +60s)` 兜底按 PR 反馈彻底删除,理由是它会重新引入 #795 这类"trace 一直在推但被外层 cap 误杀"的回归。
- **activity 来源**`store.recordActivity()``App.tsx` 串到 `<CommandBar onTyping>`+ `submitMessage` submit kickoff + `pollRunnerTrace` 在新 snapshot 时 mutate ref。`runner-trace.ts` 不直接调 `updateActivity`,避免与 `state/workbench.ts``submitActivityRef` 重复。
- **Runaway 保护责任**:用户在 Web 上有 cancel / steer / 关 tab 三个明确逃生口;CLI 上有 `--timeout-ms` 让调用方自行决定。**不**在浏览器侧加隐式硬上限。
不变量(用于 #795 / 未来回归测试):
1. `bun run --cwd web/hwlab-cloud-web check` 全过:`bun run scripts/tsc-check.ts` 严格 React TSX 0 explicit any`bun test` 5 pass / 0 fail`scripts/fetchJson-inactivity.test.ts` 3 case + dist-contract 2 case)。
2. `scripts/fetchJson-inactivity.test.ts` 必须包含 "持续 activity > inactivity 窗口" 用例:例如 200ms cadence 推 activity 共 10sinactivity 窗口 3srequest 必须活到 10s+ 才因 activity stop 而 abort。`elapsed >= 10000` 是不变量;不变量失效即视为本节契约被破坏。
3. live `http://74.48.78.17:19666/app.js` 不能含 `Math.max(t*4``while (...)``totalTimeoutMs``>=` 比较、`> 120` 之类的硬上限 pattern;如出现即视为本节契约被破坏。
历史与收敛(蒸馏自 #775 / #777 / #791 / #795 的过程):
- 迁移前 `app-device-pod.ts:fetchJson` 走的是 inactivity-timeout`scheduleTimeout` 每秒重算 `remainingMs = timeoutMs - (now - lastActivityAt)`),`waitForAgentMessageResult` 也按 `idleMs >= CODE_AGENT_TIMEOUT_MS` 走。React 迁移后 #777`client.ts` 加了 `ActivityRef` 抽象,但 `workbench.ts` / `runner-trace.ts` 没把 ref 串起来,于是 Web UI 退化成 total-timeout#795 出现"bootshar 9.7s 跑完但 Web 37s 处 timeout"的回归。
- #795 PR #797 修了一版,但留了 `max(totalTimeoutMs * 4, totalTimeoutMs + 60s)` 兜底 cap。用户反馈"硬上限等于又引入 #795 同一类回归",第二轮把 `for(;;)` + 删 `attempt > TRACE_HARD_CAP_ATTEMPTS` + 删 `startedAt` 累计计时 + per-poll 传 `totalTimeoutMs`(不缩小),实现"完全无 total / 轮询上限"。
- Round 10 起 commit / spec / 测试同步固化为本节。
- `web/hwlab-cloud-web/app.ts` 是浏览器端主入口,和 `app-device-pod.ts``app-conversation.ts``app-trace.ts``app-helpers.ts` 共同组成实际 bundle 输入集合,组织 Workbench 状态、Code Agent 会话缓存、trace 轮询和 device-pod 面板。
- `internal/dev-entrypoint/http.mjs` 提供静态服务、health 和 HTTP proxy 基础能力。
- `internal/dev-entrypoint/cloud-web-routes.mjs` 定义可代理到 cloud-api 的同源 API route 和认证边界。
@@ -82,28 +82,32 @@ test("fetchJson inactivityactivityRef 每 100ms 推一次活动,stop 后 ~2
});
test("fetchJson inactivitytrace poll 风格的 1.5s 间隔在 6.5s 内不会触发总超时", { timeout: 20000 }, async () => {
// HWLAB #795 regression: the runner-trace poll loop used a total timeout
// equal to the user-configured codeAgentTimeoutMs, so an operation that
// kept emitting events every 1.5s would still be killed by the outer
// total cap. With activityRef, the inactivity window resets on every
// snapshot so the request only aborts after timeoutMs of NO activity.
// HWLAB #795 regression + PR feedback: the runner-trace poll loop must
// NOT use any client-side total timeout. The fetchJson request aborts
// only on inactivity (no activity for timeoutMs). To make the contract
// explicit, this case runs 200ms-cadence activity for 10 seconds with
// a 3-second inactivity window set on the request — the request must
// keep living for the full 10 seconds even though 3s << 10s.
const { ref, update } = buildMutableActivityRef(Date.now());
const originalFetch = globalThis.fetch;
globalThis.fetch = bindSignalFetch();
const interval = setInterval(() => {
update(Date.now());
}, 1500);
}, 200);
try {
const { fetchJson } = await import("../src/services/api/client.ts");
const startedAt = Date.now();
const promise = fetchJson("/v1/agent/chat/result/abc", { timeoutMs: 5000, timeoutName: "trace-poll", activityRef: () => ref });
setTimeout(() => clearInterval(interval), 6500);
const promise = fetchJson("/v1/agent/chat/result/abc", { timeoutMs: 3000, timeoutName: "trace-poll", activityRef: () => ref });
setTimeout(() => clearInterval(interval), 10_000);
const result = await promise;
const elapsed = Date.now() - startedAt;
assert.equal(result.ok, false, "fetch should fail once activity stops");
assert.ok(elapsed >= 6000, `continuous 1.5s activity should keep the request alive past the per-poll inactivity window; got ${elapsed}ms`);
assert.ok(
elapsed >= 10_000,
`continuous 200ms activity must keep the request alive for the full 10s; a total cap or wall-clock shrink would abort earlier; got ${elapsed}ms`
);
assert.match(result.error ?? "", //u, `expected inactivity-timeout error, got: ${result.error}`);
} finally {
clearInterval(interval);
+21 -19
View File
@@ -23,7 +23,6 @@ export interface TraceSnapshot {
}
const TRACE_POLL_INTERVAL_MS = 1500;
const TRACE_HARD_CAP_ATTEMPTS = 120;
export function isTerminalStatus(status: string | undefined): boolean {
if (!status) return false;
@@ -99,26 +98,32 @@ export async function waitForAgentResult(
return isTerminalStatus(initial.status) ? initial : null;
}
if (isTerminalStatus(initial.status)) return initial;
const startedAt = Date.now();
// Hard ceiling is a safety net for runaway operations; the per-poll
// inactivity-timeout in fetchJson (driven by activityRef) is the primary
// abort signal so active sessions are not falsely killed.
const hardCapMs = Math.max(totalTimeoutMs * 4, totalTimeoutMs + 60_000);
let attempt = 0;
while (Date.now() - startedAt < hardCapMs) {
attempt += 1;
// No total-timeout: the per-poll inactivity-timeout in fetchJson (driven
// by activityRef) is the sole abort signal. Hard caps were removed
// because they reintroduced the HWLAB #795 regression class — a long
// operation that keeps emitting trace events could still be killed by
// the outer `while` ceiling even with an active activity ref. The user
// can still cancel / steer / use the session button; runaway protection
// is the caller's responsibility, not a browser-side hard cap. See
// docs/reference/spec-v02-hwlab-cloud-web.md "Code Agent Timeout Model".
// TRACE_HARD_CAP_ATTEMPTS is intentionally not used here: there is no
// client-side ceiling on how many times we may poll, only on how long
// a single fetchJson may go without activity. If the upstream never
// returns a terminal status and never goes silent, the loop continues
// until the user cancels / navigates away / the tab is closed.
for (;;) {
await new Promise<void>((resolve) => window.setTimeout(resolve, TRACE_POLL_INTERVAL_MS));
if (Date.now() - startedAt >= hardCapMs) break;
const polled = await api.getAgentChatResult(
initial.resultUrl,
Math.min(8000, Math.max(4000, totalTimeoutMs - (Date.now() - startedAt))),
// Pass the full totalTimeoutMs so the per-poll window is a pure
// inactivity budget; do not shrink it as wall-clock time passes.
totalTimeoutMs,
activityRef ?? null
);
if (polled.ok && polled.data) {
if (isTerminalStatus(polled.data.status)) return polled.data;
if (attempt > TRACE_HARD_CAP_ATTEMPTS) return null;
} else if (!polled.ok && polled.status >= 500) {
// server hiccup; keep polling until totalTimeoutMs
// server hiccup; keep polling — only the inactivity window can abort
} else if (!polled.ok) {
return null;
}
@@ -133,19 +138,16 @@ export async function pollRunnerTrace(
totalTimeoutMs: number,
activityRef?: ActivityRefSource
): Promise<TraceSnapshot | null> {
const startedAt = Date.now();
const hardCapMs = Math.max(totalTimeoutMs * 4, totalTimeoutMs + 60_000);
const traceUrl = traceUrlFromResultUrl(resultUrl, traceId);
let lastEventCount = -1;
let lastStatus: string | null = null;
let lastUpdatedAt: string | null = null;
while (Date.now() - startedAt < hardCapMs) {
// See waitForAgentResult above — pure inactivity-timeout, no total cap.
for (;;) {
await new Promise<void>((resolve) => window.setTimeout(resolve, TRACE_POLL_INTERVAL_MS));
if (Date.now() - startedAt >= hardCapMs) break;
const remaining = totalTimeoutMs - (Date.now() - startedAt);
const polled = await api.getAgentChatResult(
traceUrl,
Math.min(8000, Math.max(3000, remaining)),
totalTimeoutMs,
activityRef ?? null
);
if (!polled.ok || !polled.data) continue;