import assert from "node:assert/strict"; import { mkdtemp, rm } from "node:fs/promises"; import { createServer as createHttpServer } from "node:http"; import os from "node:os"; import path from "node:path"; import { test } from "bun:test"; import { createCloudApiServer } from "./server.ts"; import { createCodeAgentTraceStore } from "./code-agent-trace-store.ts"; import { loadPersistedAgentRunResult } from "./code-agent-agentrun-adapter.ts"; import { codexStdioChatFixture, codexStdioReadyFixture } from "./server-test-helpers.ts"; test("cloud api accepts user-billing API keys and records Code Agent billing usage", async () => { const calls = []; const userBillingClient = { configured: true, async introspect(token) { calls.push({ op: "introspect", tokenPrefix: token.slice(0, 8) }); assert.equal(token, "hwl_user_billing_test_secret"); return { ok: true, status: 200, body: { active: true, principal: { userId: "usr_user_billing_agent", email: "agent@hwlab.local", username: "billing-agent", role: "user", scopes: ["api"], authType: "api-key", keyId: "key_user_billing_agent" } } }; }, async billingPreflight(body) { calls.push({ op: "preflight", body }); assert.equal(body.apiKey, "hwl_user_billing_test_secret"); assert.equal(body.serviceId, "hwlab-code-agent"); assert.equal(body.estimatedCredits, 1); assert.equal(body.idempotencyKey, "code-agent:trc_user_billing_agent:preflight"); assert.equal(body.metadata.resource, "code-agent-turn"); return { ok: true, status: 200, body: { allowed: true, reservationId: "res_user_billing_agent", estimatedCredits: 1, expiresAt: "2026-06-13T12:00:00.000Z" } }; }, async billingRecord(body) { calls.push({ op: "record", body }); assert.equal(body.reservationId, "res_user_billing_agent"); assert.equal(body.serviceId, "hwlab-code-agent"); assert.equal(body.usedCredits, 1); assert.equal(body.idempotencyKey, "code-agent:trc_user_billing_agent:record"); return { ok: true, status: 200, body: { recordId: "use_user_billing_agent", credits: 1, balance: 99 } }; }, async billingSummary(token, options = {}) { calls.push({ op: "summary", tokenPrefix: token.slice(0, 8), options }); assert.equal(token, "hwl_user_billing_test_secret"); assert.equal(options.limit, 20); return { ok: true, status: 200, body: { contractVersion: "user-billing-summary-v1", status: "ok", credits: { balance: 99, reserved: 0, available: 99 }, usage: { totalCredits: 1, totalQuantity: 1200, recordCount: 1, byService: [{ serviceId: "hwlab-code-agent", credits: 1, quantity: 1200, recordCount: 1, lastUsedAt: "2026-06-13T12:00:00.000Z" }] }, ledger: { count: 1, limit: 20, rows: [{ id: "led_user_billing_agent", kind: "usage", reason: "hwlab-code-agent", deltaCredits: -1, balanceAfter: 99, createdAt: "2026-06-13T12:00:00.000Z" }] }, reservations: { activeCount: 0, active: [] }, state: { stateless: true, stateAuthority: "pk01-postgres", redis: { role: "cache-only" } }, valuesRedacted: true } }; }, async adminBillingSummary(options = {}) { calls.push({ op: "adminSummary", options }); assert.equal(options.limit, 50); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-summary-v1", status: "ok", count: 1, limit: 50, totals: { users: 1, activeUsers: 1, balanceCredits: 99, usageCredits: 1, activeReservations: 0 }, users: [{ user: { id: "usr_user_billing_agent", email: "agent@hwlab.local", username: "billing-agent", displayName: "Billing Agent", role: "user", status: "active" }, planId: "default", credits: { balance: 99, reserved: 0, available: 99 }, usage: { totalCredits: 1, totalQuantity: 1200, recordCount: 1, lastUsedAt: "2026-06-13T12:00:00.000Z" }, apiKeys: { activeCount: 1, totalCount: 1, lastUsedAt: "2026-06-13T12:00:00.000Z" }, reservations: { activeCount: 0, active: [] } }], state: { stateless: true, stateAuthority: "pk01-postgres", redis: { role: "cache-only" } }, valuesRedacted: true } }; } }; const workspace = await mkdtemp(path.join(os.tmpdir(), "hwlab-user-billing-agent-")); const codexHome = await mkdtemp(path.join(os.tmpdir(), "hwlab-user-billing-agent-codex-")); const server = createCloudApiServer({ env: { PATH: process.env.PATH, CODEX_HOME: codexHome, HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_USER_BILLING_CODE_AGENT_ENABLED: "1", HWLAB_CODE_AGENT_CODEX_WORKSPACE: workspace, HWLAB_CODE_AGENT_CODEX_SANDBOX: "danger-full-access", HWLAB_CODE_AGENT_PROVIDER: "codex-stdio", HWLAB_CODE_AGENT_MODEL: "gpt-test", OPENAI_API_KEY: "test-openai-key-material", HWLAB_CODE_AGENT_OPENAI_BASE_URL: "http://127.0.0.1:65535/v1/responses" }, userBillingClient, codexStdioManager: { describe() { return { ...codexStdioReadyFixture({ workspace, codexHome }), sandbox: "danger-full-access" }; }, async probe() { return { ...codexStdioReadyFixture({ workspace, codexHome }), sandbox: "danger-full-access" }; }, async chat(params = {}) { return { ...codexStdioChatFixture({ workspace, codexHome, params }), usage: { totalTokens: 1200 }, sandbox: "danger-full-access" }; }, cancel() {}, reapIdle() {} } }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const authHeader = { authorization: "Bearer hwl_user_billing_test_secret" }; const me = await fetch(`http://127.0.0.1:${port}/v1/users/me`, { headers: authHeader }); assert.equal(me.status, 200); const meBody = await me.json(); assert.equal(meBody.actor.id, "usr_user_billing_agent"); assert.equal(meBody.authMethod, "user-billing-api-key"); const sessionResponse = await fetch(`http://127.0.0.1:${port}/v1/agent/sessions`, { method: "POST", headers: { "content-type": "application/json", ...authHeader }, body: JSON.stringify({ conversationId: "cnv_user_billing_agent", sessionId: "ses_user_billing_agent", providerProfile: "deepseek" }) }); assert.equal(sessionResponse.status, 201); const manual = await sessionResponse.json(); assert.equal(manual.ok, true); assert.equal(manual.session.sessionId, "ses_user_billing_agent"); const submit = await fetch(`http://127.0.0.1:${port}/v1/agent/chat`, { method: "POST", headers: { "content-type": "application/json", authorization: "Bearer hwl_user_billing_test_secret", "x-trace-id": "trc_user_billing_agent", prefer: "respond-async" }, body: JSON.stringify({ conversationId: manual.session.conversationId, sessionId: manual.session.sessionId, shortConnection: true, message: "billing integration smoke" }) }); assert.equal(submit.status, 202); const result = await pollUserBillingAgentResult(port, "trc_user_billing_agent", authHeader); assert.equal(result.status, "completed"); assert.equal(result.billing.recorded, true); assert.equal(result.billing.reservationId, "res_user_billing_agent"); assert.equal(result.billing.recordId, "use_user_billing_agent"); const summaryResponse = await fetch(`http://127.0.0.1:${port}/v1/usage/summary`, { headers: authHeader }); assert.equal(summaryResponse.status, 200); const summary = await summaryResponse.json(); assert.equal(summary.contractVersion, "user-billing-summary-v1"); assert.equal(summary.credits.available, 99); assert.equal(summary.usage.byService[0].serviceId, "hwlab-code-agent"); assert.equal(summary.state.stateAuthority, "pk01-postgres"); assert.equal(summary.state.redis.role, "cache-only"); assert.equal(summary.proxy.source, "hwlab-user-billing"); assert.equal(JSON.stringify(summary).includes("hwl_user_billing_test_secret"), false); const adminSummaryDenied = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/summary`, { headers: authHeader }); assert.equal(adminSummaryDenied.status, 403); const adminStatusDenied = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_user_billing_agent/status`, { method: "PATCH", headers: { "content-type": "application/json", ...authHeader }, body: JSON.stringify({ status: "disabled" }) }); assert.equal(adminStatusDenied.status, 403); assert.deepEqual(calls.map((call) => call.op), ["introspect", "introspect", "introspect", "preflight", "record", "introspect", "introspect", "summary", "introspect", "introspect"]); assert.equal(JSON.stringify(result).includes("hwl_user_billing_test_secret"), false); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); await rm(workspace, { recursive: true, force: true }); await rm(codexHome, { recursive: true, force: true }); } }); test("AgentRun persisted trace evidence restores billing reservation without exposing it on agentRun", async () => { const restored = await loadPersistedAgentRunResult("trc_user_billing_agentrun_restored", { env: { HWLAB_CODE_AGENT_ADAPTER: "agentrun-v01" }, accessController: { async getAgentSessionByTraceId(traceId) { assert.equal(traceId, "trc_user_billing_agentrun_restored"); return { id: "ses_user_billing_agentrun_restored", ownerUserId: "usr_user_billing_agentrun", ownerRole: "user", conversationId: "cnv_user_billing_agentrun", threadId: "thr_billing_restored", status: "running", startedAt: "2026-06-14T04:00:00.000Z", updatedAt: "2026-06-14T04:10:00.000Z", session: { traceResults: { trc_user_billing_agentrun_restored: { traceId: "trc_user_billing_agentrun_restored", status: "running", userBillingReservation: { reservationId: "res_user_billing_restored", estimatedCredits: 1, expiresAt: "2026-06-14T05:00:00.000Z", valuesRedacted: true, secretMaterialStored: false }, agentRun: { adapter: "agentrun-v01", managerUrl: "http://agentrun-mgr.agentrun-v02.svc.cluster.local:8080", backendProfile: "deepseek", providerId: "D601", runId: "run_billing_restored", commandId: "cmd_billing_restored", status: "running", commandState: "running", traceId: "trc_user_billing_agentrun_restored", valuesPrinted: false } } } } }; } } }); assert.equal(restored.userBillingReservation.reservationId, "res_user_billing_restored"); assert.equal(restored.agentRun.runId, "run_billing_restored"); assert.equal(Object.hasOwn(restored.agentRun, "userBillingReservation"), false); }); async function pollUserBillingAgentResult(port, traceId, headers) { let last = null; for (let attempt = 0; attempt < 20; attempt += 1) { const response = await fetch(`http://127.0.0.1:${port}/v1/agent/chat/result/${encodeURIComponent(traceId)}`, { headers }); last = { status: response.status, body: await response.json() }; if (response.status === 200) return last.body; await new Promise((resolve) => setTimeout(resolve, 10)); } throw new Error(`Code Agent result did not complete: ${JSON.stringify(last)}`); } async function waitForUserBillingCondition(predicate, timeoutMs = 500) { const deadline = Date.now() + timeoutMs; while (Date.now() < deadline) { if (predicate()) return; await new Promise((resolve) => setTimeout(resolve, 10)); } throw new Error("condition was not met before timeout"); } test("cloud api proxies admin billing summary for web admins without exposing user-billing secrets", async () => { const calls = []; const userBillingClient = { configured: true, async adminBillingSummary(options = {}) { calls.push({ op: "adminSummary", options }); assert.equal(options.limit, 50); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-summary-v1", status: "ok", count: 1, limit: 50, totals: { users: 1, activeUsers: 1, balanceCredits: 25, reservedCredits: 5, availableCredits: 20, usageCredits: 3, activeReservations: 1 }, users: [{ user: { id: "usr_billing_admin_subject", email: "subject@hwlab.local", username: "billing-subject", displayName: "Billing Subject", role: "user", status: "active" }, planId: "default", credits: { balance: 25, reserved: 5, available: 20 }, usage: { totalCredits: 3, totalQuantity: 2048, recordCount: 2, lastUsedAt: "2026-06-13T13:00:00.000Z" }, apiKeys: { activeCount: 1, totalCount: 2, lastUsedAt: "2026-06-13T12:30:00.000Z" }, reservations: { activeCount: 1, active: [{ reservationId: "res_admin_billing", serviceId: "hwlab-hwpod", estimatedCredits: 5, estimatedTokens: 0, status: "reserved", expiresAt: "2026-06-13T13:30:00.000Z", createdAt: "2026-06-13T13:00:00.000Z" }] } }], state: { stateless: true, stateAuthority: "pk01-postgres", redis: { role: "cache-only" } }, valuesRedacted: true } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_BOOTSTRAP_ADMIN_USERNAME: "admin", HWLAB_BOOTSTRAP_ADMIN_PASSWORD: "admin-password-1127" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "admin-password-1127" }) }); assert.equal(login.status, 200); const cookie = login.headers.get("set-cookie")?.split(";")[0] ?? ""; assert.match(cookie, /^hwlab_session=/u); const response = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/summary`, { headers: { cookie } }); assert.equal(response.status, 200); const summary = await response.json(); assert.equal(summary.contractVersion, "user-billing-admin-summary-v1"); assert.equal(summary.users[0].user.id, "usr_billing_admin_subject"); assert.equal(summary.users[0].credits.available, 20); assert.equal(summary.users[0].reservations.active[0].serviceId, "hwlab-hwpod"); assert.equal(summary.state.stateAuthority, "pk01-postgres"); assert.equal(summary.state.redis.role, "cache-only"); assert.equal(summary.proxy.source, "hwlab-user-billing"); assert.equal(JSON.stringify(summary).includes("admin-password-1127"), false); assert.deepEqual(calls.map((call) => call.op), ["adminSummary"]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api proxies admin user status updates for web admins", async () => { const calls = []; const userBillingClient = { configured: true, async updateAdminUserStatus(userId, status) { calls.push({ op: "adminUserStatus", userId, status }); assert.equal(userId, "usr_billing_admin_subject"); assert.equal(status, "disabled"); return { ok: true, status: 200, body: { userId, status } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_BOOTSTRAP_ADMIN_USERNAME: "admin", HWLAB_BOOTSTRAP_ADMIN_PASSWORD: "admin-password-1127" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "admin-password-1127" }) }); assert.equal(login.status, 200); const cookie = login.headers.get("set-cookie")?.split(";")[0] ?? ""; assert.match(cookie, /^hwlab_session=/u); const invalid = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_billing_admin_subject/status`, { method: "PATCH", headers: { "content-type": "application/json", cookie }, body: JSON.stringify({ status: "archived" }) }); assert.equal(invalid.status, 400); const response = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_billing_admin_subject/status`, { method: "PATCH", headers: { "content-type": "application/json", cookie }, body: JSON.stringify({ status: "disabled" }) }); assert.equal(response.status, 200); const payload = await response.json(); assert.equal(payload.userId, "usr_billing_admin_subject"); assert.equal(payload.status, "disabled"); assert.equal(payload.proxy.source, "hwlab-user-billing"); assert.equal(payload.proxy.route, "/v1/admin/billing/users/{userId}/status"); assert.equal(payload.valuesRedacted, true); assert.equal(JSON.stringify(payload).includes("admin-password-1127"), false); assert.deepEqual(calls, [{ op: "adminUserStatus", userId: "usr_billing_admin_subject", status: "disabled" }]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api proxies R1 admin user management and credit adjustment to user-billing", async () => { const calls = []; const userBillingClient = { configured: true, async adminUsers(options = {}) { calls.push({ op: "adminUsers", options }); assert.equal(options.page, 2); assert.equal(options.pageSize, 10); assert.equal(options.search, "billing"); assert.equal(options.status, "active"); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-users-v1", users: [], count: 0, total: 0, pagination: { page: 2, pageSize: 10 } } }; }, async adminUser(userId) { calls.push({ op: "adminUser", userId }); assert.equal(userId, "usr_billing_admin_subject"); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-user-v1", detail: { summary: { user: { id: userId, username: "billing-subject" } } } } }; }, async createAdminUser(body = {}) { calls.push({ op: "createAdminUser", body }); assert.equal(body.email, "new@hwlab.local"); return { ok: true, status: 201, body: { contractVersion: "user-billing-admin-user-v1", created: true, user: { id: "usr_created", email: body.email } } }; }, async updateAdminUser(userId, body = {}) { calls.push({ op: "updateAdminUser", userId, body }); assert.equal(userId, "usr_billing_admin_subject"); assert.equal(body.displayName, "Updated Subject"); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-user-v1", updated: true, user: { id: userId, displayName: body.displayName } } }; }, async adjustAdminCredits(body = {}) { calls.push({ op: "adjustAdminCredits", body }); assert.equal(body.userId, "usr_billing_admin_subject"); assert.equal(body.deltaCredits, 25); assert.equal(body.metadata.operator.username, "admin"); return { ok: true, status: 200, body: { userId: body.userId, balance: 124 } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_BOOTSTRAP_ADMIN_USERNAME: "admin", HWLAB_BOOTSTRAP_ADMIN_PASSWORD: "admin-password-1127" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "admin-password-1127" }) }); assert.equal(login.status, 200); const cookie = login.headers.get("set-cookie")?.split(";")[0] ?? ""; const adminHeaders = { "content-type": "application/json", cookie }; const list = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users?page=2&pageSize=10&search=billing&status=active`, { headers: { cookie } }); assert.equal(list.status, 200); assert.equal((await list.json()).proxy.source, "hwlab-user-billing"); const detail = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_billing_admin_subject`, { headers: { cookie } }); assert.equal(detail.status, 200); assert.equal((await detail.json()).detail.summary.user.id, "usr_billing_admin_subject"); const created = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users`, { method: "POST", headers: adminHeaders, body: JSON.stringify({ email: "new@hwlab.local", username: "new-user", password: "new-password-1127" }) }); assert.equal(created.status, 201); const updated = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_billing_admin_subject`, { method: "PATCH", headers: adminHeaders, body: JSON.stringify({ displayName: "Updated Subject" }) }); assert.equal(updated.status, 200); const adjusted = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_billing_admin_subject/credits/adjust`, { method: "POST", headers: adminHeaders, body: JSON.stringify({ deltaCredits: 25, reason: "manual top-up" }) }); assert.equal(adjusted.status, 200); assert.equal((await adjusted.json()).balance, 124); assert.equal(JSON.stringify(calls).includes("admin-password-1127"), false); assert.deepEqual(calls.map((call) => call.op), ["adminUsers", "adminUser", "createAdminUser", "updateAdminUser", "adjustAdminCredits"]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api proxies R4 admin billing plans and preserves plan updates", async () => { const calls = []; const userBillingClient = { configured: true, async adminBillingPlans() { calls.push({ op: "adminBillingPlans" }); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-plans-v1", plans: [{ plan: { id: "default", displayName: "Default", status: "active" }, entitlements: [{ resourceType: "code_agent", enabled: true, monthlyQuotaUnlimited: true, concurrencyUnlimited: true, rpmUnlimited: true }] }], count: 1, valuesRedacted: true } }; }, async updateAdminUser(userId, body = {}) { calls.push({ op: "updateAdminUser", userId, body }); assert.equal(userId, "usr_billing_admin_subject"); assert.equal(body.planId, "default"); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-user-v1", updated: true, detail: { summary: { user: { id: userId }, planId: body.planId } } } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_BOOTSTRAP_ADMIN_USERNAME: "admin", HWLAB_BOOTSTRAP_ADMIN_PASSWORD: "admin-password-1127" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "admin-password-1127" }) }); assert.equal(login.status, 200); const cookie = login.headers.get("set-cookie")?.split(";")[0] ?? ""; const plans = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/plans`, { headers: { cookie } }); assert.equal(plans.status, 200); const plansBody = await plans.json(); assert.equal(plansBody.proxy.source, "hwlab-user-billing"); assert.equal(plansBody.plans[0].entitlements[0].resourceType, "code_agent"); const update = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/users/usr_billing_admin_subject`, { method: "PATCH", headers: { "content-type": "application/json", cookie }, body: JSON.stringify({ planId: "default" }) }); assert.equal(update.status, 200); assert.equal((await update.json()).detail.summary.planId, "default"); assert.deepEqual(calls.map((call) => call.op), ["adminBillingPlans", "updateAdminUser"]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api maps R4 Code Agent entitlement denial to a user-facing async failure", async () => { const userBillingClient = { configured: true, async introspect(token) { assert.equal(token, "hwl_user_billing_entitlement_denied"); return { ok: true, status: 200, body: { active: true, principal: { userId: "usr_denied", email: "denied@hwlab.local", username: "denied", role: "user", scopes: ["api"], authType: "api-key", keyId: "key_denied" } } }; }, async billingPreflight(body) { assert.equal(body.serviceId, "hwlab-code-agent"); return { ok: false, status: 403, error: { code: "resource_not_entitled", message: "resource is not enabled for this billing plan" } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_USER_BILLING_CODE_AGENT_ENABLED: "1" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const session = await fetch(`http://127.0.0.1:${port}/v1/agent/sessions`, { method: "POST", headers: { "content-type": "application/json", authorization: "Bearer hwl_user_billing_entitlement_denied" }, body: JSON.stringify({ conversationId: "cnv_denied", sessionId: "ses_denied", providerProfile: "deepseek" }) }); assert.equal(session.status, 201); const response = await fetch(`http://127.0.0.1:${port}/v1/agent/chat`, { method: "POST", headers: { "content-type": "application/json", authorization: "Bearer hwl_user_billing_entitlement_denied", "x-trace-id": "trc_user_billing_entitlement_denied", prefer: "respond-async" }, body: JSON.stringify({ conversationId: "cnv_denied", sessionId: "ses_denied", shortConnection: true, message: "should be blocked by entitlement" }) }); assert.equal(response.status, 202); const accepted = await response.json(); assert.equal(accepted.accepted, true); assert.equal(accepted.traceId, "trc_user_billing_entitlement_denied"); const payload = await pollUserBillingAgentResult(port, "trc_user_billing_entitlement_denied", { authorization: "Bearer hwl_user_billing_entitlement_denied" }); assert.equal(payload.status, "failed"); assert.equal(payload.error.code, "resource_not_entitled"); assert.equal(payload.blocker.code, "resource_not_entitled"); assert.equal(payload.blocker.retryable, false); assert.match(payload.finalResponse.text, /resource is not enabled/u); assert.equal(JSON.stringify(payload).includes("hwl_user_billing_entitlement_denied"), false); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api proxies R1 user-billing API key and profile self-service routes", async () => { const calls = []; const bearer = "hwl_user_billing_session_secret"; const userBillingClient = { configured: true, async introspect(token) { calls.push({ op: "introspect", tokenPrefix: token.slice(0, 8) }); assert.equal(token, bearer); return { ok: true, status: 200, body: { active: true, principal: { userId: "usr_self_service", email: "self@hwlab.local", username: "self-service", role: "user", scopes: ["session"], authType: "session" } } }; }, async listApiKeys(token) { calls.push({ op: "listApiKeys", tokenPrefix: token.slice(0, 8) }); return { ok: true, status: 200, body: { contractVersion: "user-api-key-v1", keys: [{ id: "key_self", prefix: "hwl_self", status: "active" }], count: 1 } }; }, async billingSummary(token, query = {}) { calls.push({ op: "billingSummary", tokenPrefix: token.slice(0, 8), query }); return { ok: true, status: 200, body: { contractVersion: "user-billing-summary-v1", credits: { balance: 5, reserved: 1, available: 4 }, usage: { totalCredits: 0, recordCount: 0 }, valuesRedacted: true } }; }, async createApiKey(token, body = {}) { calls.push({ op: "createApiKey", tokenPrefix: token.slice(0, 8), body }); assert.equal(body.name, "Self key"); return { ok: true, status: 201, body: { contractVersion: "user-api-key-v1", apiKey: { id: "key_created", prefix: "hwl_created" }, secret: "hwl_created_secret", created: true } }; }, async updateApiKey(token, keyId, body = {}) { calls.push({ op: "updateApiKey", tokenPrefix: token.slice(0, 8), keyId, body }); assert.equal(keyId, "key_created"); assert.equal(body.name, "Renamed key"); return { ok: true, status: 200, body: { contractVersion: "user-api-key-v1", key: { id: keyId, name: body.name }, updated: true } }; }, async revokeApiKey(token, keyId) { calls.push({ op: "revokeApiKey", tokenPrefix: token.slice(0, 8), keyId }); assert.equal(keyId, "key_created"); return { ok: true, status: 200, body: { contractVersion: "user-api-key-v1", key: { id: keyId, status: "revoked" }, revoked: true } }; }, async updateProfile(token, body = {}) { calls.push({ op: "updateProfile", tokenPrefix: token.slice(0, 8), body }); assert.equal(body.displayName, "Self Service"); assert.equal("username" in body, false); return { ok: true, status: 200, body: { contractVersion: "user-profile-v1", updated: true, user: { id: "usr_self_service", displayName: body.displayName } } }; }, async changePassword(token, body = {}) { calls.push({ op: "changePassword", tokenPrefix: token.slice(0, 8), bodyKeys: Object.keys(body).sort() }); assert.equal(body.currentPassword, "old-password-1127"); return { ok: true, status: 200, body: { contractVersion: "user-profile-v1", changed: true } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const headers = { authorization: `Bearer ${bearer}`, "content-type": "application/json" }; const list = await fetch(`http://127.0.0.1:${port}/v1/api-keys`, { headers }); assert.equal(list.status, 200); assert.equal((await list.json()).proxy.source, "hwlab-user-billing"); const billingSummary = await fetch(`http://127.0.0.1:${port}/v1/billing/summary?limit=7`, { headers }); assert.equal(billingSummary.status, 200); const billingSummaryBody = await billingSummary.json(); assert.equal(billingSummaryBody.contractVersion, "user-billing-summary-v1"); assert.equal(billingSummaryBody.proxy.route, "/v1/billing/summary"); const create = await fetch(`http://127.0.0.1:${port}/v1/api-keys`, { method: "POST", headers, body: JSON.stringify({ name: "Self key" }) }); assert.equal(create.status, 201); const update = await fetch(`http://127.0.0.1:${port}/v1/api-keys/key_created`, { method: "PATCH", headers, body: JSON.stringify({ name: "Renamed key" }) }); assert.equal(update.status, 200); const revoke = await fetch(`http://127.0.0.1:${port}/v1/api-keys/key_created`, { method: "DELETE", headers, body: JSON.stringify({}) }); assert.equal(revoke.status, 200); const profile = await fetch(`http://127.0.0.1:${port}/v1/users/me/profile`, { method: "PATCH", headers, body: JSON.stringify({ displayName: "Self Service" }) }); assert.equal(profile.status, 200); const password = await fetch(`http://127.0.0.1:${port}/v1/users/me/password`, { method: "POST", headers, body: JSON.stringify({ currentPassword: "old-password-1127", newPassword: "new-password-1127" }) }); assert.equal(password.status, 200); assert.equal(JSON.stringify(calls).includes(bearer), false); assert.deepEqual(calls.map((call) => call.op), ["introspect", "listApiKeys", "introspect", "billingSummary", "introspect", "createApiKey", "introspect", "updateApiKey", "introspect", "revokeApiKey", "introspect", "updateProfile", "introspect", "changePassword"]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api bridges user-billing register/login into HttpOnly web session cookie", async () => { const calls = []; const sessionToken = "hws_user_billing_web_session_secret"; const userBillingClient = { configured: true, async register(body = {}) { calls.push({ op: "register", body }); assert.equal(body.email, "new-user@hwlab.local"); assert.equal(body.username, "new-user"); assert.equal(body.password, "new-password-1188"); return { ok: true, status: 201, body: { user: { id: "usr_web_new", email: body.email, username: body.username, role: "user", status: "active" }, initialCredits: 0 } }; }, async login(body = {}) { calls.push({ op: "login", username: body.username, email: body.email, passwordSeen: Boolean(body.password) }); assert.equal(body.password, "new-password-1188"); return { ok: true, status: 200, body: { token: sessionToken, tokenType: "Bearer", user: { id: "usr_web_new", email: "new-user@hwlab.local", username: "new-user", role: "user", status: "active" } } }; }, async introspect(token) { calls.push({ op: "introspect", tokenPrefix: token.slice(0, 8) }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { active: true, principal: { userId: "usr_web_new", email: "new-user@hwlab.local", username: "new-user", role: "user", scopes: ["session"], authType: "session" } } }; }, async listApiKeys(token) { calls.push({ op: "listApiKeys", tokenPrefix: token.slice(0, 8) }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { contractVersion: "user-api-key-v1", keys: [{ id: "key_web_session", name: "Web session key", prefix: "hwl_web" }], count: 1 } }; }, async billingSummary(token, query = {}) { calls.push({ op: "billingSummary", tokenPrefix: token.slice(0, 8), query }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { contractVersion: "user-billing-summary-v1", planId: "default", entitlements: [{ resourceType: "code_agent", enabled: true }], valuesRedacted: true } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const register = await fetch(`http://127.0.0.1:${port}/auth/register`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ email: "new-user@hwlab.local", username: "new-user", displayName: "New User", password: "new-password-1188" }) }); assert.equal(register.status, 201); const cookie = register.headers.get("set-cookie")?.split(";")[0] ?? ""; assert.match(cookie, /^hwlab_session=hws_/u); const body = await register.json(); assert.equal(body.authMethod, "user-billing-session"); assert.equal(body.actor.username, "new-user"); assert.equal(JSON.stringify(body).includes(sessionToken), false); const session = await fetch(`http://127.0.0.1:${port}/auth/session`, { headers: { cookie } }); assert.equal(session.status, 200); const sessionBody = await session.json(); assert.equal(sessionBody.authMethod, "user-billing-session"); assert.equal(sessionBody.actor.id, "usr_web_new"); assert.equal(sessionBody.actor.username, "new-user"); assert.equal(JSON.stringify(sessionBody).includes(sessionToken), false); const billingSummary = await fetch(`http://127.0.0.1:${port}/v1/billing/summary?limit=3`, { headers: { cookie } }); assert.equal(billingSummary.status, 200); const billingSummaryBody = await billingSummary.json(); assert.equal(billingSummaryBody.planId, "default"); assert.equal(billingSummaryBody.entitlements[0].resourceType, "code_agent"); assert.equal(billingSummaryBody.proxy.source, "hwlab-user-billing"); assert.equal(JSON.stringify(billingSummaryBody).includes(sessionToken), false); const keys = await fetch(`http://127.0.0.1:${port}/v1/api-keys`, { headers: { cookie } }); assert.equal(keys.status, 200); const keysBody = await keys.json(); assert.equal(keysBody.proxy.source, "hwlab-user-billing"); assert.equal(keysBody.keys[0].id, "key_web_session"); assert.equal(JSON.stringify(keysBody).includes(sessionToken), false); assert.deepEqual(calls.map((call) => call.op), ["register", "login", "introspect", "introspect", "introspect", "billingSummary", "introspect", "listApiKeys"]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api falls back to local bootstrap login when user-billing login is transient", async () => { const calls = []; const userBillingClient = { configured: true, async login(body = {}) { calls.push({ op: "login", username: body.username, passwordSeen: Boolean(body.password) }); return { ok: false, status: 503, retryable: true, retryCount: 2, transientObserved: true, error: { code: "user_billing_unreachable", message: "The operation timed out." } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1", HWLAB_BOOTSTRAP_ADMIN_USERNAME: "admin", HWLAB_BOOTSTRAP_ADMIN_PASSWORD: "local-admin-password-1353" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "local-admin-password-1353" }) }); assert.equal(login.status, 200); assert.match(login.headers.get("set-cookie") ?? "", /hwlab_session=/u); const body = await login.json(); assert.equal(body.actor.username, "admin"); const wrongLocalPassword = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "wrong-local-password" }) }); assert.equal(wrongLocalPassword.status, 401); const nonLocal = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "billing-only", password: "billing-only-password" }) }); assert.equal(nonLocal.status, 503); const error = await nonLocal.json(); assert.equal(error.error.code, "user_billing_unreachable"); assert.equal(error.error.retryable, true); assert.equal(error.error.dependency.serviceId, "hwlab-user-billing"); assert.equal(error.error.dependency.retryCount, 2); assert.equal(error.error.dependency.transientObserved, true); assert.equal(calls.length, 3); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api reports auth_required before user-billing profile/password proxy config errors", async () => { const userBillingClient = { configured: true, async updateProfile() { throw new Error("profile proxy should not run without auth"); }, async changePassword() { throw new Error("password proxy should not run without auth"); } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const profile = await fetch(`http://127.0.0.1:${port}/v1/users/me/profile`, { method: "PATCH", headers: { "content-type": "application/json" }, body: JSON.stringify({ displayName: "Missing Auth" }) }); assert.equal(profile.status, 401); assert.equal((await profile.json()).error.code, "auth_required"); const password = await fetch(`http://127.0.0.1:${port}/v1/users/me/password`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ currentPassword: "old-password", newPassword: "new-password" }) }); assert.equal(password.status, 401); assert.equal((await password.json()).error.code, "auth_required"); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api proxies R6 redeem subscription payment and admin redeem routes", async () => { const calls = []; const sessionToken = "hws_r6_redeem_session_secret"; const userBillingClient = { configured: true, async login(body = {}) { calls.push({ op: "login", username: body.username }); assert.equal(body.username, "admin"); assert.equal(body.password, "admin-r6-password"); return { ok: true, status: 200, body: { token: sessionToken, user: { id: "usr_r6_admin", username: "admin", role: "admin", status: "active" } } }; }, async introspect(token) { calls.push({ op: "introspect", tokenPrefix: token.slice(0, 8) }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { active: true, principal: { userId: "usr_r6_admin", username: "admin", email: "admin@hwlab.local", role: "admin", scopes: ["session"], authType: "session" } } }; }, async redeem(token, body = {}) { calls.push({ op: "redeem", tokenPrefix: token.slice(0, 8), body }); assert.equal(token, sessionToken); assert.equal(body.code, "hwl_redeem_test"); return { ok: true, status: 200, body: { contractVersion: "user-billing-redeem-v1", result: { redeemed: true, creditAmount: 7, balanceBefore: 10, balanceAfter: 17, source: "redeem", ledgerId: "led_r6" } } }; }, async subscriptionSummary(token) { calls.push({ op: "subscriptionSummary", tokenPrefix: token.slice(0, 8) }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { contractVersion: "user-billing-subscription-v1", planId: "default", payment: { configured: false, status: "unconfigured" } } }; }, async paymentSummary(token) { calls.push({ op: "paymentSummary", tokenPrefix: token.slice(0, 8) }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { contractVersion: "user-billing-payment-v1", provider: { configured: false, status: "unconfigured" }, orders: [] } }; }, async adminRedeemCodes(options = {}) { calls.push({ op: "adminRedeemCodes", options }); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-redeem-codes-v1", codes: [{ id: "rdc_r6", codePrefix: "hwl_redeem", status: "active", creditAmount: 7 }], count: 1 } }; }, async createAdminRedeemCode(body = {}) { calls.push({ op: "createAdminRedeemCode", body }); assert.equal(body.creditAmount, 7); assert.equal(body.createdByUsername, "admin"); return { ok: true, status: 201, body: { contractVersion: "user-billing-admin-redeem-codes-v1", code: { id: "rdc_r6" }, redeemCode: "hwl_redeem_created", created: true } }; }, async updateAdminRedeemCode(codeId, body = {}) { calls.push({ op: "updateAdminRedeemCode", codeId, body }); assert.equal(codeId, "rdc_r6"); assert.equal(body.status, "disabled"); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-redeem-codes-v1", code: { id: codeId, status: body.status }, updated: true } }; }, async adminRedeemRedemptions(options = {}) { calls.push({ op: "adminRedeemRedemptions", options }); assert.equal(options.codeId, "rdc_r6"); return { ok: true, status: 200, body: { contractVersion: "user-billing-admin-redeem-redemptions-v1", redemptions: [{ id: "rdm_r6", codeId: "rdc_r6", ledgerId: "led_r6", creditAmount: 7, status: "applied" }], count: 1 } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ username: "admin", password: "admin-r6-password" }) }); assert.equal(login.status, 200); const cookie = login.headers.get("set-cookie")?.split(";")[0] ?? ""; const redeem = await fetch(`http://127.0.0.1:${port}/v1/redeem`, { method: "POST", headers: { "content-type": "application/json", cookie }, body: JSON.stringify({ code: "hwl_redeem_test" }) }); assert.equal(redeem.status, 200); assert.equal((await redeem.json()).result.source, "redeem"); const subscription = await fetch(`http://127.0.0.1:${port}/v1/subscription/summary`, { headers: { cookie } }); assert.equal(subscription.status, 200); assert.equal((await subscription.json()).payment.status, "unconfigured"); const payment = await fetch(`http://127.0.0.1:${port}/v1/payments/summary`, { headers: { cookie } }); assert.equal(payment.status, 200); assert.equal((await payment.json()).provider.configured, false); const codes = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/redeem-codes?status=active`, { headers: { cookie } }); assert.equal(codes.status, 200); assert.equal((await codes.json()).codes[0].id, "rdc_r6"); const created = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/redeem-codes`, { method: "POST", headers: { "content-type": "application/json", cookie }, body: JSON.stringify({ creditAmount: 7, reason: "campaign" }) }); assert.equal(created.status, 201); assert.equal((await created.json()).created, true); const disabled = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/redeem-codes/rdc_r6`, { method: "PATCH", headers: { "content-type": "application/json", cookie }, body: JSON.stringify({ status: "disabled" }) }); assert.equal(disabled.status, 200); assert.equal((await disabled.json()).code.status, "disabled"); const redemptions = await fetch(`http://127.0.0.1:${port}/v1/admin/billing/redeem-redemptions?codeId=rdc_r6`, { headers: { cookie } }); assert.equal(redemptions.status, 200); assert.equal((await redemptions.json()).redemptions[0].ledgerId, "led_r6"); assert.equal(JSON.stringify(calls).includes(sessionToken), false); for (const op of ["redeem", "subscriptionSummary", "paymentSummary", "adminRedeemCodes", "createAdminRedeemCode", "updateAdminRedeemCode", "adminRedeemRedemptions"]) { assert.equal(calls.some((call) => call.op === op), true, op); } } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } }); test("cloud api exposes auth login OTel trace headers and forwards traceparent", async () => { const calls = []; const sessionToken = "hws_otel_login_session_secret"; const userBillingClient = { configured: true, async login(body = {}, options = {}) { calls.push({ op: "login", username: body.username, traceparent: options.traceparent }); return { ok: true, status: 200, body: { token: sessionToken, user: { id: "usr_otel_admin", username: "admin", role: "admin", status: "active" } } }; }, async introspect(token) { calls.push({ op: "introspect", tokenPrefix: token.slice(0, 8) }); assert.equal(token, sessionToken); return { ok: true, status: 200, body: { active: true, principal: { userId: "usr_otel_admin", username: "admin", email: "admin@hwlab.local", role: "admin", scopes: ["session"], authType: "session" } } }; } }; const server = createCloudApiServer({ env: { HWLAB_ACCESS_CONTROL_REQUIRED: "1" }, userBillingClient }); await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); try { const { port } = server.address(); const upstreamTraceparent = "00-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbbb-01"; const login = await fetch(`http://127.0.0.1:${port}/auth/login`, { method: "POST", headers: { "content-type": "application/json", traceparent: upstreamTraceparent, "x-request-id": "req_auth_otel_login" }, body: JSON.stringify({ username: "admin", password: "redacted-password" }) }); assert.equal(login.status, 200); const responseTraceparent = login.headers.get("traceparent") ?? ""; assert.match(responseTraceparent, /^00-[0-9a-f]{32}-[0-9a-f]{16}-01$/u); assert.equal(login.headers.get("x-hwlab-otel-trace-id"), "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"); assert.equal(login.headers.get("x-request-id"), "req_auth_otel_login"); assert.equal(responseTraceparent.split("-")[1], "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"); const loginCall = calls.find((call) => call.op === "login"); assert.match(loginCall?.traceparent ?? "", /^00-[0-9a-f]{32}-[0-9a-f]{16}-01$/u); assert.equal(loginCall.traceparent.split("-")[1], "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"); assert.notEqual(loginCall.traceparent.split("-")[2], responseTraceparent.split("-")[2]); } finally { await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); } });