Merge pull request #2813 from pikasTech/feat/2811-root-cgroup-file-cache-reclaim
为 NC01 增加 root cgroup 文件缓存受控回收
This commit is contained in:
@@ -199,6 +199,8 @@ bun scripts/cli.ts gc remote <providerId> status --job-id <id>
|
|||||||
```bash
|
```bash
|
||||||
bun scripts/cli.ts gc remote <node> plan --memory-pressure-only
|
bun scripts/cli.ts gc remote <node> plan --memory-pressure-only
|
||||||
bun scripts/cli.ts gc remote <node> run --confirm --memory-pressure-only
|
bun scripts/cli.ts gc remote <node> run --confirm --memory-pressure-only
|
||||||
|
bun scripts/cli.ts gc remote <node> plan --memory-pressure-only --memory-reclaim-config-id <id>
|
||||||
|
bun scripts/cli.ts gc remote <node> run --confirm --memory-pressure-only --memory-reclaim-config-id <id>
|
||||||
bun scripts/cli.ts gc remote <node> status --job-id <id>
|
bun scripts/cli.ts gc remote <node> status --job-id <id>
|
||||||
bun scripts/cli.ts gc remote <node> memory
|
bun scripts/cli.ts gc remote <node> memory
|
||||||
```
|
```
|
||||||
@@ -211,6 +213,8 @@ bun scripts/cli.ts gc remote <node> memory
|
|||||||
- job 未终态时不得提前执行 `memory` 或重试 WebProbe;
|
- job 未终态时不得提前执行 `memory` 或重试 WebProbe;
|
||||||
- 终态后执行 `memory`,只有不再命中 owning YAML 的 `manualComparator` 时才可重试人工启动;
|
- 终态后执行 `memory`,只有不再命中 owning YAML 的 `manualComparator` 时才可重试人工启动;
|
||||||
- 内存处置不得改用未带 `--memory-pressure-only` 的通用 plan/run。
|
- 内存处置不得改用未带 `--memory-pressure-only` 的通用 plan/run。
|
||||||
|
- 只执行一个 cgroup 回收目标时,plan/run 必须携带相同的 `--memory-reclaim-config-id <id>`;
|
||||||
|
- `<id>` 只能来自 owning YAML `memoryPressure.cgroupReclaim.targets[].id`,不得按运行面身份或候选排序猜测。
|
||||||
- sentinel cadence:
|
- sentinel cadence:
|
||||||
- 命中 owning YAML 的 `sentinelComparator` 时返回 `skipped-wait-next-round`;
|
- 命中 owning YAML 的 `sentinelComparator` 时返回 `skipped-wait-next-round`;
|
||||||
- 不创建 observer、不启动 Chromium、不自动执行 GC;
|
- 不创建 observer、不启动 Chromium、不自动执行 GC;
|
||||||
|
|||||||
@@ -223,7 +223,7 @@ gc:
|
|||||||
minAgeHours: 336
|
minAgeHours: 336
|
||||||
maxDeletePerRun: 5
|
maxDeletePerRun: 5
|
||||||
memoryPressure:
|
memoryPressure:
|
||||||
planPreviewLimit: 4
|
planPreviewLimit: 2
|
||||||
processPatterns:
|
processPatterns:
|
||||||
- k3s-server
|
- k3s-server
|
||||||
- containerd
|
- containerd
|
||||||
@@ -291,6 +291,14 @@ gc:
|
|||||||
targetMemAvailableBytes: 3221225472
|
targetMemAvailableBytes: 3221225472
|
||||||
minimumSwapFreeBytesAfterReclaim: 34359738368
|
minimumSwapFreeBytesAfterReclaim: 34359738368
|
||||||
targets:
|
targets:
|
||||||
|
- id: root-cgroup-file-cache
|
||||||
|
rootCgroupSelector:
|
||||||
|
scope: root
|
||||||
|
reclaimBytes: 805306368
|
||||||
|
swappiness: 0
|
||||||
|
minimumInactiveFileBytes: 268435456
|
||||||
|
minimumReclaimableFileSlabBytes: 1073741824
|
||||||
|
priority: 2
|
||||||
- id: k3s-service-inactive-anon
|
- id: k3s-service-inactive-anon
|
||||||
systemdUnitSelector:
|
systemdUnitSelector:
|
||||||
unit: k3s.service
|
unit: k3s.service
|
||||||
|
|||||||
@@ -28,6 +28,8 @@
|
|||||||
- `--full` 只用于下钻完整候选身份明细,不作为 scoped run 的前置步骤;
|
- `--full` 只用于下钻完整候选身份明细,不作为 scoped run 的前置步骤;
|
||||||
- 不得混入通用磁盘 GC 候选;
|
- 不得混入通用磁盘 GC 候选;
|
||||||
- 执行入口必须显式携带 `--confirm`。
|
- 执行入口必须显式携带 `--confirm`。
|
||||||
|
- 需要只执行一个 cgroup 回收策略时,plan 和 run 必须同时携带 `--memory-reclaim-config-id <id>`;
|
||||||
|
- `<id>` 只匹配 owning YAML `memoryPressure.cgroupReclaim.targets[].id`,精确模式不收集或执行其他 cgroup 与进程候选。
|
||||||
- `gc remote <providerId> memory`:
|
- `gc remote <providerId> memory`:
|
||||||
- 只读读取 `/proc/meminfo` 的 `MemAvailable`;
|
- 只读读取 `/proc/meminfo` 的 `MemAvailable`;
|
||||||
- 不执行 cluster preflight、候选扫描或 mutation;
|
- 不执行 cluster preflight、候选扫描或 mutation;
|
||||||
@@ -232,6 +234,8 @@ JD01 Web observe artifact 是一等 GC 对象。state root 必须来自 YAML;
|
|||||||
```bash
|
```bash
|
||||||
bun scripts/cli.ts gc remote <node> plan --memory-pressure-only
|
bun scripts/cli.ts gc remote <node> plan --memory-pressure-only
|
||||||
bun scripts/cli.ts gc remote <node> run --confirm --memory-pressure-only
|
bun scripts/cli.ts gc remote <node> run --confirm --memory-pressure-only
|
||||||
|
bun scripts/cli.ts gc remote <node> plan --memory-pressure-only --memory-reclaim-config-id <id>
|
||||||
|
bun scripts/cli.ts gc remote <node> run --confirm --memory-pressure-only --memory-reclaim-config-id <id>
|
||||||
bun scripts/cli.ts gc remote <node> status --job-id <id>
|
bun scripts/cli.ts gc remote <node> status --job-id <id>
|
||||||
bun scripts/cli.ts gc remote <node> memory
|
bun scripts/cli.ts gc remote <node> memory
|
||||||
```
|
```
|
||||||
@@ -240,6 +244,7 @@ bun scripts/cli.ts gc remote <node> memory
|
|||||||
- 先复核 `plan --memory-pressure-only` 的候选、保护对象、预计回收量和 `runEligibility`;
|
- 先复核 `plan --memory-pressure-only` 的候选、保护对象、预计回收量和 `runEligibility`;
|
||||||
- 只有 `runEligibility.allowed=true` 时才允许继续;
|
- 只有 `runEligibility.allowed=true` 时才允许继续;
|
||||||
- 只执行参数完全匹配的 `run --confirm --memory-pressure-only`;
|
- 只执行参数完全匹配的 `run --confirm --memory-pressure-only`;
|
||||||
|
- 精确 cgroup plan 使用 `--memory-reclaim-config-id` 时,run 必须携带同一个 YAML `id`;
|
||||||
- `run` 返回 job id 后,重复执行 `status --job-id <id>` 直到明确终态;
|
- `run` 返回 job id 后,重复执行 `status --job-id <id>` 直到明确终态;
|
||||||
- job 未终态时,禁止提前执行 `memory` 或重试 WebProbe;
|
- job 未终态时,禁止提前执行 `memory` 或重试 WebProbe;
|
||||||
- job 终态后执行 `memory`,以新的 `/proc/meminfo` 快照重新判定;
|
- job 终态后执行 `memory`,以新的 `/proc/meminfo` 快照重新判定;
|
||||||
|
|||||||
@@ -72,7 +72,7 @@ YAML运维负责 HWLAB/UniDesk 自有平台配置的真相源、解析、渲染
|
|||||||
| host proxy client | 新节点 k3s 安装前由 `trans` 分发的 0 依赖出网代理客户端,用于给 Docker、k3s、containerd、包管理器、Git 和 git-mirror 提供初始出网。 |
|
| host proxy client | 新节点 k3s 安装前由 `trans` 分发的 0 依赖出网代理客户端,用于给 Docker、k3s、containerd、包管理器、Git 和 git-mirror 提供初始出网。 |
|
||||||
| 配置组合 | 通过 baseline、configRef、继承、覆盖和变量渲染生成有效计划的配置方式;组合结果是执行视图,不是第二份可编辑真相源。 |
|
| 配置组合 | 通过 baseline、configRef、继承、覆盖和变量渲染生成有效计划的配置方式;组合结果是执行视图,不是第二份可编辑真相源。 |
|
||||||
| 稳定工作负载选择器 | 由 namespace 和 Kubernetes labels 组成、跨 Pod 重建保持不变的工作负载身份;Pod UID 只能是运行面解析结果。 |
|
| 稳定工作负载选择器 | 由 namespace 和 Kubernetes labels 组成、跨 Pod 重建保持不变的工作负载身份;Pod UID 只能是运行面解析结果。 |
|
||||||
| 精确 cgroup 回收 | 受控 CLI 将稳定工作负载选择器解析为当前 Running Pod cgroup,并在执行前重新解析身份后请求有界 `memory.reclaim`。 |
|
| 精确 cgroup 回收 | 受控 CLI 将稳定工作负载或 root cgroup 选择器解析为当前 cgroup,并在执行前重新解析身份后请求有界 `memory.reclaim`。 |
|
||||||
|
|
||||||
## 4. 系统边界和接口
|
## 4. 系统边界和接口
|
||||||
|
|
||||||
@@ -249,8 +249,12 @@ YAML运维状态输出应区分三类依赖:必须保留的 host bootstrap 面
|
|||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| OPS-YAML-REQ-012 | 精确工作负载回收 | PJ2026-01060315 精确工作负载回收 | [平台运维](PJ2026-0106-platform-ops.md)、[NC01 CI资源治理](PJ2026-01060107-nc01-ci-resource-governance.md) |
|
| OPS-YAML-REQ-012 | 精确工作负载回收 | PJ2026-01060315 精确工作负载回收 | [平台运维](PJ2026-0106-platform-ops.md)、[NC01 CI资源治理](PJ2026-01060107-nc01-ci-resource-governance.md) |
|
||||||
|
|
||||||
平台 GC 应允许 owning YAML 使用稳定工作负载选择器声明精确 cgroup 回收目标。Kubernetes workload selector 至少包含 namespace、非空 labels 和最大匹配 Pod 数;host service 使用仅包含稳定 unit 名的 `systemdUnitSelector`。目标可用内存、单目标回收上限、swappiness、inactive anonymous memory 下限、swap 保留线和优先级继续由 YAML 声明。YAML 不得保存 Pod UID、容器 ID、MainPID、进程 startTicks、当前 cgroup 路径、cgroup inode 或从运行面反解出的可调数值。
|
平台 GC 应允许 owning YAML 使用稳定工作负载选择器声明精确 cgroup 回收目标。Kubernetes workload selector 至少包含 namespace、非空 labels 和最大匹配 Pod 数;host service 使用仅包含稳定 unit 名的 `systemdUnitSelector`。root cgroup 文件页回收使用唯一的 `rootCgroupSelector`,不得用任意路径选择器代替。目标可用内存、单目标回收上限、swappiness、inactive anonymous memory 下限、文件页回收下限、swap 保留线和优先级继续由 YAML 声明。YAML 不得保存 Pod UID、容器 ID、MainPID、进程 startTicks、当前 cgroup 路径、cgroup inode 或从运行面反解出的可调数值。
|
||||||
|
|
||||||
|
受控 CLI 应允许操作人员按 owning YAML 中唯一且稳定的 cgroup 回收目标 `id` 精确生成 plan 和 run。精确选择不得从当前 Pod、进程、cgroup 路径或候选排序推断目标;同一次 run 只能执行该 `id` 解析出的候选,不得在其之前、之后或并行执行其他 cgroup 回收目标或进程回收候选。
|
||||||
|
|
||||||
受控 plan 对 Kubernetes selector 只选择 Running Pod,并披露 selector、解析出的 namespace、Pod、UID、cgroup、当前内存、inactive anonymous memory 和保护原因。对 `systemdUnitSelector` 必须通过 `systemctl show` 读取 `LoadState`、`ActiveState`、`SubState`、`ControlGroup` 和 `MainPID`,只接受 loaded、active、running 的 service;ControlGroup 必须唯一、绝对、位于 `/system.slice`,映射到 `/sys/fs/cgroup/system.slice` 下的真实非 symlink 目录,且 MainPID 当前 cgroup 必须与 ControlGroup 一致。plan 同时记录 MainPID 的 `/proc/<pid>/stat` startTicks,禁止从 unit 名猜测路径或回退到父 cgroup。
|
受控 plan 对 Kubernetes selector 只选择 Running Pod,并披露 selector、解析出的 namespace、Pod、UID、cgroup、当前内存、inactive anonymous memory 和保护原因。对 `systemdUnitSelector` 必须通过 `systemctl show` 读取 `LoadState`、`ActiveState`、`SubState`、`ControlGroup` 和 `MainPID`,只接受 loaded、active、running 的 service;ControlGroup 必须唯一、绝对、位于 `/system.slice`,映射到 `/sys/fs/cgroup/system.slice` 下的真实非 symlink 目录,且 MainPID 当前 cgroup 必须与 ControlGroup 一致。plan 同时记录 MainPID 的 `/proc/<pid>/stat` startTicks,禁止从 unit 名猜测路径或回退到父 cgroup。
|
||||||
|
|
||||||
执行前必须使用同一 YAML selector 重新解析当前目标。Kubernetes workload 逐项核对 Pod UID 与 cgroup 路径;systemd service 逐项核对 unit、`LoadState`、`ActiveState`、`SubState`、`ControlGroup`、MainPID 与 startTicks;两类目标都必须重新核对全部回收参数。身份变化、目标已满足或候选不再合格时跳过且结构化披露;只有身份闭合的当前 cgroup 才能写入有界 `memory.reclaim`。执行不得停止、重启、驱逐或迁移 workload,也不得把内核请求上限宣称为实际收益;终态继续分别披露执行前后 cgroup current、宿主 `MemAvailable` 变化、真实收益、目标缺口和 swap 排除状态。
|
root cgroup plan 必须确认 YAML 声明的 cgroup 根是当前 cgroup v2 mount 的根目录,并记录 mount 身份和目录 inode;任意 symlink、非 cgroup v2、非 mount root 或身份读取失败都必须保护。文件页候选固定使用 `swappiness=0`,并同时读取 `/proc/meminfo` 与 root `memory.stat` 的 `file`、`inactive_file` 和 `slab_reclaimable` 证据。候选只有在宿主与 root cgroup 的文件页及可回收 slab 上界都达到 YAML 下限,且每项必要证据完整时才可进入 plan。请求字节必须标记为 `kernel-request-upper-bound`,不得把文件页或 slab 观测量宣称为确定收益。
|
||||||
|
|
||||||
|
执行前必须使用同一 YAML selector 重新解析当前目标。Kubernetes workload 逐项核对 Pod UID 与 cgroup 路径;systemd service 逐项核对 unit、`LoadState`、`ActiveState`、`SubState`、`ControlGroup`、MainPID 与 startTicks;root cgroup 逐项核对 cgroup v2 mount 身份、目录 inode、`/proc/meminfo`、root `memory.stat` 和全部文件页下限。各类目标都必须重新核对全部回收参数。身份变化、`MemAvailable` 大于等于 YAML 目标、证据变化后低于下限或候选不再合格时跳过且结构化披露;只有身份闭合的当前 cgroup 才能写入有界 `memory.reclaim`。root 文件页回收不得使用非零 swappiness、全局 `drop_caches`、workload 停止或重启。终态必须分别披露执行前后宿主 `MemAvailable`、root `memory.stat`、真实增量、剩余目标缺口和内核错误;compact 与 full 终态都必须保留这些判定证据。内核请求上限、文件页上界和 Swap 都不得计入实际收益。
|
||||||
|
|||||||
@@ -465,7 +465,8 @@ def collect_memory_distribution(observed_at):
|
|||||||
"swapExcludedFromAvailability": True,
|
"swapExcludedFromAvailability": True,
|
||||||
"availableBytes": available,
|
"availableBytes": available,
|
||||||
"targetAvailableBytes": target,
|
"targetAvailableBytes": target,
|
||||||
"targetGapBytes": max(0, target - available + 1),
|
"targetMet": target > 0 and available >= target,
|
||||||
|
"targetGapBytes": max(0, target - available),
|
||||||
"anonPagesBytes": meminfo.get("AnonPages"),
|
"anonPagesBytes": meminfo.get("AnonPages"),
|
||||||
"cachedAndBuffersBytes": cached,
|
"cachedAndBuffersBytes": cached,
|
||||||
"slabBytes": meminfo.get("Slab"),
|
"slabBytes": meminfo.get("Slab"),
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ function runPythonFixture(body: string): Record<string, unknown> {
|
|||||||
const fixture = [
|
const fixture = [
|
||||||
"import json, os, re, signal, time",
|
"import json, os, re, signal, time",
|
||||||
"PROVIDER_ID = 'NC01'",
|
"PROVIDER_ID = 'NC01'",
|
||||||
|
"OPTIONS = {}",
|
||||||
"def safe_int(value, fallback=0):",
|
"def safe_int(value, fallback=0):",
|
||||||
" try: return int(value)",
|
" try: return int(value)",
|
||||||
" except (TypeError, ValueError): return fallback",
|
" except (TypeError, ValueError): return fallback",
|
||||||
@@ -34,6 +35,22 @@ function runPythonFixture(body: string): Record<string, unknown> {
|
|||||||
return JSON.parse(result.stdout.trim()) as Record<string, unknown>;
|
return JSON.parse(result.stdout.trim()) as Record<string, unknown>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function runRunnerPythonFixture(functionNames: string[], body: string): Record<string, unknown> {
|
||||||
|
const source = Buffer.from(runnerSource, "utf8").toString("base64");
|
||||||
|
const fixture = [
|
||||||
|
"import ast, base64, json, os, re, time",
|
||||||
|
`source = base64.b64decode(${JSON.stringify(source)}).decode("utf-8")`,
|
||||||
|
`wanted = set(${JSON.stringify(functionNames)})`,
|
||||||
|
"tree = ast.parse(source)",
|
||||||
|
"selected = [node for node in tree.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name in wanted]",
|
||||||
|
"exec(compile(ast.Module(body=selected, type_ignores=[]), '<gc-remote-runner-fixture>', 'exec'))",
|
||||||
|
body,
|
||||||
|
].join("\n");
|
||||||
|
const result = spawnSync("python3", ["-"], { input: fixture, encoding: "utf8" });
|
||||||
|
assert.equal(result.status, 0, result.stderr || result.stdout);
|
||||||
|
return JSON.parse(result.stdout.trim()) as Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
test("memory-pressure candidate fixtures protect active observers and classify only closed stale/orphan trees", () => {
|
test("memory-pressure candidate fixtures protect active observers and classify only closed stale/orphan trees", () => {
|
||||||
const result = runPythonFixture(String.raw`
|
const result = runPythonFixture(String.raw`
|
||||||
MEMORY_CONFIG = {
|
MEMORY_CONFIG = {
|
||||||
@@ -200,6 +217,354 @@ print(json.dumps({"ids": [item["id"] for item in candidates], "swappiness": [ite
|
|||||||
assert.match(webObserveSource, /memory\.reclaim/u);
|
assert.match(webObserveSource, /memory\.reclaim/u);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("root cgroup file-cache reclaim requires cgroup2 identity and fresh dual-source lower bounds", () => {
|
||||||
|
const result = runPythonFixture(String.raw`
|
||||||
|
MEMORY_CONFIG = {
|
||||||
|
"observeStateRoots": ["/tmp/observe"], "minimumReadableObserveRoots": 1,
|
||||||
|
"observeRunDepth": 6, "observeScanLimit": 100, "staleRunMaxAgeHours": 1,
|
||||||
|
"processScanLimit": 100, "processStaleMinAgeSeconds": 3600,
|
||||||
|
"maxMemoryReclaimCandidates": 20, "terminationGraceSeconds": 1,
|
||||||
|
"terminationPollMilliseconds": 50, "observerRootPatterns": ["observer-runner.mjs"],
|
||||||
|
"browserRootPatterns": ["cliDaemon.mjs"], "browserProcessPatterns": ["chromium"],
|
||||||
|
"attribution": {"cgroupRoot": "/sys/fs/cgroup", "commandTimeoutSeconds": 20},
|
||||||
|
"cgroupReclaim": {
|
||||||
|
"enabled": True, "targetMemAvailableBytes": 4294967296,
|
||||||
|
"minimumSwapFreeBytesAfterReclaim": 402653184,
|
||||||
|
"targets": [{
|
||||||
|
"id": "root-file-cache", "rootCgroupSelector": {"scope": "root"},
|
||||||
|
"reclaimBytes": 805306368, "swappiness": 0,
|
||||||
|
"minimumInactiveFileBytes": 268435456,
|
||||||
|
"minimumReclaimableFileSlabBytes": 1073741824,
|
||||||
|
"priority": 1,
|
||||||
|
}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
identity = {
|
||||||
|
"filesystemType": "cgroup2", "mountRoot": "/", "mountPoint": "/sys/fs/cgroup",
|
||||||
|
"majorMinor": "0:29", "mountId": 31, "parentMountId": 30, "inode": 1,
|
||||||
|
"path": "/sys/fs/cgroup",
|
||||||
|
}
|
||||||
|
root_stat = {
|
||||||
|
"anon": 1024, "file": 1610612736, "inactive_anon": 0,
|
||||||
|
"inactive_file": 805306368, "slab_reclaimable": 536870912,
|
||||||
|
}
|
||||||
|
host_evidence = {
|
||||||
|
"ok": True, "source": "/proc/meminfo", "cachedBytes": 1610612736,
|
||||||
|
"buffersBytes": 134217728, "slabReclaimableBytes": 536870912,
|
||||||
|
"cachedAndBuffersBytes": 1744830464,
|
||||||
|
"reclaimableFileSlabUpperBoundBytes": 2281701376,
|
||||||
|
"estimateKind": "kernel-reclaimable-upper-bound",
|
||||||
|
}
|
||||||
|
fresh_evidence = lambda: {
|
||||||
|
"ok": True, "host": host_evidence, "rootMemoryStat": root_stat,
|
||||||
|
"rootReclaimableFileSlabUpperBoundBytes": 2147483648,
|
||||||
|
"estimateKind": "kernel-reclaimable-upper-bound",
|
||||||
|
}
|
||||||
|
read_proc_process_table = lambda: {1: {"pid": 1, "ppid": 0, "sid": 1, "startTicks": 1, "ageSeconds": 1, "rssBytes": 1, "commandLine": "init", "commandPreview": "init"}}
|
||||||
|
observer_run_records = lambda: []
|
||||||
|
collect_memory_snapshot = lambda: {"ok": True, "memory": {"availableBytes": 3 * 1024 * 1024 * 1024}, "swap": {"freeBytes": 2 * 1024 * 1024 * 1024}}
|
||||||
|
resolve_root_cgroup_identity = lambda: {"ok": True, "path": "/sys/fs/cgroup", "identity": identity}
|
||||||
|
collect_root_file_reclaim_evidence = lambda path: fresh_evidence()
|
||||||
|
read_cgroup_memory_stat = lambda path: root_stat
|
||||||
|
read_cgroup_integer = lambda path, name: 8 * 1024 * 1024 * 1024
|
||||||
|
os.path.realpath = lambda path: path
|
||||||
|
os.path.isdir = lambda path: True
|
||||||
|
os.path.islink = lambda path: False
|
||||||
|
os.path.exists = lambda path: True
|
||||||
|
os.access = lambda path, mode: True
|
||||||
|
candidates = collect_memory_reclaim_candidates()
|
||||||
|
memory_reclaim_write_attempts = []
|
||||||
|
original_open = open
|
||||||
|
def tracked_open(path, *args, **kwargs):
|
||||||
|
if str(path).endswith("/memory.reclaim"):
|
||||||
|
memory_reclaim_write_attempts.append(path)
|
||||||
|
return original_open(path, *args, **kwargs)
|
||||||
|
open = tracked_open
|
||||||
|
collect_root_file_reclaim_evidence = lambda path: {
|
||||||
|
**fresh_evidence(),
|
||||||
|
"rootMemoryStat": {**root_stat, "inactive_file": 134217728},
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
execute_cgroup_reclaim_candidate(candidates[0])
|
||||||
|
revalidation_error = None
|
||||||
|
except RuntimeError as error:
|
||||||
|
revalidation_error = str(error)
|
||||||
|
print(json.dumps({
|
||||||
|
"candidate": candidates[0],
|
||||||
|
"revalidationError": revalidation_error,
|
||||||
|
"memoryReclaimWriteAttempts": memory_reclaim_write_attempts,
|
||||||
|
"diagnostics": memory_reclaim_scan_diagnostics(),
|
||||||
|
}))
|
||||||
|
`);
|
||||||
|
const candidate = result.candidate as Record<string, any>;
|
||||||
|
assert.equal(candidate.rootCgroupSelector.scope, "root");
|
||||||
|
assert.equal(candidate.resolvedRootCgroup.filesystemType, "cgroup2");
|
||||||
|
assert.equal(candidate.resolvedRootCgroup.mountRoot, "/");
|
||||||
|
assert.equal(candidate.swappiness, 0);
|
||||||
|
assert.equal(candidate.estimateKind, "kernel-request-upper-bound");
|
||||||
|
assert.equal(candidate.rootFileReclaimEvidence.host.source, "/proc/meminfo");
|
||||||
|
assert.equal(result.revalidationError, "minimum root inactive file memory is no longer available");
|
||||||
|
assert.deepEqual(result.memoryReclaimWriteAttempts, []);
|
||||||
|
assert.equal(((result.diagnostics as Record<string, any>).cgroupReclaim).eligibleCount, 1);
|
||||||
|
const projected = projectRemoteGcResult({
|
||||||
|
ok: true,
|
||||||
|
summary: { candidateCount: 1 },
|
||||||
|
candidateScan: result.diagnostics,
|
||||||
|
candidates: [candidate],
|
||||||
|
}, { memoryPressure: { planPreviewLimit: 4 } }, { action: "plan", memoryPressureOnly: true, full: false }) as Record<string, any>;
|
||||||
|
assert.equal(projected.runEligibility.allowed, true);
|
||||||
|
assert.equal(projected.candidatePreview.candidates[0].rootCgroupSelector, "root");
|
||||||
|
assert.equal(projected.candidatePreview.candidates[0].resolvedRootCgroup.inode, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("root-only selection uses the stable YAML config id, excludes anonymous candidates, and treats equality as met", () => {
|
||||||
|
const result = runPythonFixture(String.raw`
|
||||||
|
OPTIONS = {"memoryReclaimConfigId": "root-file-cache"}
|
||||||
|
MEMORY_CONFIG = {
|
||||||
|
"attribution": {"cgroupRoot": "/sys/fs/cgroup", "commandTimeoutSeconds": 20},
|
||||||
|
"cgroupReclaim": {
|
||||||
|
"enabled": True, "targetMemAvailableBytes": 3221225472,
|
||||||
|
"minimumSwapFreeBytesAfterReclaim": 1073741824,
|
||||||
|
"targets": [
|
||||||
|
{
|
||||||
|
"id": "root-file-cache", "rootCgroupSelector": {"scope": "root"},
|
||||||
|
"reclaimBytes": 536870912, "swappiness": 0,
|
||||||
|
"minimumInactiveFileBytes": 268435456,
|
||||||
|
"minimumReclaimableFileSlabBytes": 1073741824, "priority": 2,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "anonymous-pages", "systemdUnitSelector": {"unit": "k3s.service"},
|
||||||
|
"reclaimBytes": 536870912, "swappiness": 200,
|
||||||
|
"minimumInactiveAnonBytes": 536870912, "priority": 1,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
identity = {
|
||||||
|
"filesystemType": "cgroup2", "mountRoot": "/", "mountPoint": "/sys/fs/cgroup",
|
||||||
|
"majorMinor": "0:29", "mountId": 31, "parentMountId": 30, "inode": 1,
|
||||||
|
}
|
||||||
|
root_stat = {"file": 1610612736, "inactive_file": 805306368, "slab_reclaimable": 536870912}
|
||||||
|
root_evidence = {
|
||||||
|
"ok": True,
|
||||||
|
"host": {"ok": True, "source": "/proc/meminfo", "reclaimableFileSlabUpperBoundBytes": 2147483648},
|
||||||
|
"rootMemoryStat": root_stat,
|
||||||
|
"rootReclaimableFileSlabUpperBoundBytes": 2147483648,
|
||||||
|
"estimateKind": "kernel-reclaimable-upper-bound",
|
||||||
|
}
|
||||||
|
available = 2147483648
|
||||||
|
collect_memory_snapshot = lambda: {"ok": True, "memory": {"availableBytes": available}, "swap": {"freeBytes": 4294967296}}
|
||||||
|
resolve_root_cgroup_identity = lambda: {"ok": True, "path": "/sys/fs/cgroup", "identity": identity}
|
||||||
|
collect_root_file_reclaim_evidence = lambda path: root_evidence
|
||||||
|
read_cgroup_integer = lambda path, name: 8589934592
|
||||||
|
read_cgroup_memory_stat = lambda path: root_stat
|
||||||
|
os.path.realpath = lambda path: path
|
||||||
|
os.path.isdir = lambda path: True
|
||||||
|
os.path.islink = lambda path: False
|
||||||
|
os.path.exists = lambda path: True
|
||||||
|
os.access = lambda path, mode: True
|
||||||
|
read_proc_process_table = lambda: (_ for _ in ()).throw(RuntimeError("process scan must not run for exact cgroup selection"))
|
||||||
|
below = collect_memory_reclaim_candidates()
|
||||||
|
below_diagnostics = memory_reclaim_scan_diagnostics()
|
||||||
|
available = 3221225472
|
||||||
|
equal = collect_memory_reclaim_candidates()
|
||||||
|
equal_diagnostics = memory_reclaim_scan_diagnostics()
|
||||||
|
equality_execution = execute_cgroup_reclaim_candidate(below[0])
|
||||||
|
print(json.dumps({
|
||||||
|
"belowIds": [item.get("configId") for item in below],
|
||||||
|
"belowSwappiness": [item.get("swappiness") for item in below],
|
||||||
|
"belowDiagnostics": below_diagnostics,
|
||||||
|
"equalCount": len(equal),
|
||||||
|
"equalDiagnostics": equal_diagnostics,
|
||||||
|
"equalityExecution": equality_execution,
|
||||||
|
}))
|
||||||
|
`);
|
||||||
|
assert.deepEqual(result.belowIds, ["root-file-cache"]);
|
||||||
|
assert.deepEqual(result.belowSwappiness, [0]);
|
||||||
|
const belowCgroup = (result.belowDiagnostics as Record<string, any>).cgroupReclaim;
|
||||||
|
assert.equal(belowCgroup.selectedConfigId, "root-file-cache");
|
||||||
|
assert.equal(belowCgroup.totalConfiguredCount, 2);
|
||||||
|
assert.equal(belowCgroup.eligibleCount, 1);
|
||||||
|
assert.equal(result.equalCount, 0);
|
||||||
|
const equalProtected = (result.equalDiagnostics as Record<string, any>).cgroupReclaim.protected;
|
||||||
|
assert.equal(equalProtected[0].id, "root-file-cache");
|
||||||
|
assert.equal(equalProtected[0].reason, "target-memavailable-already-met");
|
||||||
|
const equalityExecution = result.equalityExecution as Record<string, any>;
|
||||||
|
assert.equal(equalityExecution.outcome, "target-memavailable-already-met");
|
||||||
|
assert.equal(equalityExecution.kernelWriteAttempted, false);
|
||||||
|
assert.equal(equalityExecution.targetGapBeforeBytes, 0);
|
||||||
|
assert.equal(equalityExecution.rootFileReclaimEvidenceBefore.ok, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("successful memory worker terminal preserves root evidence and gaps in compact and full output", () => {
|
||||||
|
const result = runRunnerPythonFixture([
|
||||||
|
"returned_results",
|
||||||
|
"summarize_memory_candidates",
|
||||||
|
"memory_pressure_selection",
|
||||||
|
"finish_memory_pressure_job",
|
||||||
|
], String.raw`
|
||||||
|
PROVIDER_ID = "NC01"
|
||||||
|
OPTIONS = {"full": False, "resultLimit": 50, "memoryReclaimConfigId": "root-file-cache"}
|
||||||
|
target = 3221225472
|
||||||
|
MEMORY_CONFIG = {"cgroupReclaim": {"targetMemAvailableBytes": target}}
|
||||||
|
def safe_int(value, fallback=0):
|
||||||
|
try: return int(value)
|
||||||
|
except (TypeError, ValueError): return fallback
|
||||||
|
def fmt_bytes(value): return str(value)
|
||||||
|
def job_paths(job_id): return {"state": "/tmp/fixture-state"}
|
||||||
|
def status_command(job_id): return "gc remote NC01 status --job-id %s" % job_id
|
||||||
|
def now_iso(): return "2026-07-21T00:00:01Z"
|
||||||
|
def update_memory_pressure_job_progress(*args): pass
|
||||||
|
written = []
|
||||||
|
def write_memory_job_state(paths, payload): written.append(payload)
|
||||||
|
snapshots = []
|
||||||
|
def collect_memory_snapshot(): return snapshots.pop(0)
|
||||||
|
before_evidence = {"ok": True, "rootMemoryStat": {"file": 20, "inactive_file": 10, "slab_reclaimable": 5}}
|
||||||
|
after_evidence = {"ok": True, "rootMemoryStat": {"file": 15, "inactive_file": 5, "slab_reclaimable": 4}}
|
||||||
|
executed = []
|
||||||
|
def execute(candidate):
|
||||||
|
executed.append(candidate.get("configId"))
|
||||||
|
return {
|
||||||
|
"status": "succeeded", "outcome": "partial-reclaim", "reclaimedMemoryBytes": 100,
|
||||||
|
"hostMemAvailableDeltaBytes": 100, "cgroupMemoryDeltaBytes": 80,
|
||||||
|
"cgroupMemoryBeforeBytes": 1000, "cgroupMemoryAfterBytes": 920,
|
||||||
|
"kernelWriteAttempted": True, "kernelWriteError": None,
|
||||||
|
"memoryBefore": {"memory": {"availableBytes": target - 100}},
|
||||||
|
"memoryAfter": {"memory": {"availableBytes": target}},
|
||||||
|
"rootFileReclaimEvidenceBefore": before_evidence,
|
||||||
|
"rootFileReclaimEvidenceAfter": after_evidence,
|
||||||
|
"targetGapBeforeBytes": 100, "targetGapAfterBytes": 0,
|
||||||
|
}
|
||||||
|
candidate = {
|
||||||
|
"id": "cgroup-memory-reclaim:root-file-cache", "configId": "root-file-cache",
|
||||||
|
"kind": "cgroup-memory-reclaim", "reclaimBytes": 100, "swappiness": 0,
|
||||||
|
"fixtureFullOnly": "kept-only-in-full",
|
||||||
|
}
|
||||||
|
snapshots[:] = [
|
||||||
|
{"ok": True, "memory": {"availableBytes": target - 100}},
|
||||||
|
{"ok": True, "memory": {"availableBytes": target}},
|
||||||
|
]
|
||||||
|
finish_memory_pressure_job("compact", [candidate], "2026-07-21T00:00:00Z", 10, 20)
|
||||||
|
compact = written[-1]
|
||||||
|
OPTIONS["full"] = True
|
||||||
|
snapshots[:] = [
|
||||||
|
{"ok": True, "memory": {"availableBytes": target - 100}},
|
||||||
|
{"ok": True, "memory": {"availableBytes": target}},
|
||||||
|
]
|
||||||
|
finish_memory_pressure_job("full", [candidate], "2026-07-21T00:00:00Z", 10, 20)
|
||||||
|
full = written[-1]
|
||||||
|
print(json.dumps({"compact": compact, "full": full, "executed": executed}))
|
||||||
|
`);
|
||||||
|
const compact = result.compact as Record<string, any>;
|
||||||
|
const full = result.full as Record<string, any>;
|
||||||
|
assert.deepEqual(result.executed, ["root-file-cache", "root-file-cache"]);
|
||||||
|
assert.deepEqual(compact.results[0].rootFileReclaimEvidenceBefore.rootMemoryStat, { file: 20, inactive_file: 10, slab_reclaimable: 5 });
|
||||||
|
assert.deepEqual(compact.results[0].rootFileReclaimEvidenceAfter.rootMemoryStat, { file: 15, inactive_file: 5, slab_reclaimable: 4 });
|
||||||
|
assert.equal(compact.results[0].targetGapBeforeBytes, 100);
|
||||||
|
assert.equal(compact.results[0].targetGapAfterBytes, 0);
|
||||||
|
assert.equal(compact.results[0].fixtureFullOnly, undefined);
|
||||||
|
assert.equal(full.results[0].fixtureFullOnly, "kept-only-in-full");
|
||||||
|
assert.equal(full.results[0].targetGapAfterBytes, 0);
|
||||||
|
assert.equal(compact.summary.targetMet, true);
|
||||||
|
assert.equal(compact.summary.targetGapBytes, 0);
|
||||||
|
assert.equal(compact.summary.outcome, "memory-target-met");
|
||||||
|
assert.equal(compact.selection.configId, "root-file-cache");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("memory-pressure plan and run treat an exact target with zero candidates as a successful no-op", () => {
|
||||||
|
const result = runRunnerPythonFixture([
|
||||||
|
"summarize_memory_candidates",
|
||||||
|
"apply_memory_target_summary",
|
||||||
|
"memory_pressure_selection",
|
||||||
|
"memory_pressure_option_suffix",
|
||||||
|
"plan_payload",
|
||||||
|
"start_memory_pressure_job",
|
||||||
|
], String.raw`
|
||||||
|
PROVIDER_ID = "NC01"
|
||||||
|
OPTIONS = {"memoryPressureOnly": True, "memoryReclaimConfigId": "root-file-cache"}
|
||||||
|
target = 3221225472
|
||||||
|
MEMORY_CONFIG = {"cgroupReclaim": {"targetMemAvailableBytes": target}}
|
||||||
|
def safe_int(value, fallback=0):
|
||||||
|
try: return int(value)
|
||||||
|
except (TypeError, ValueError): return fallback
|
||||||
|
def fmt_bytes(value): return str(value)
|
||||||
|
def now_iso(): return "2026-07-21T00:00:01Z"
|
||||||
|
def status_command(job_id): return "gc remote NC01 status --job-id %s" % job_id
|
||||||
|
snapshot = {"ok": True, "memory": {"availableBytes": target}, "swap": {"freeBytes": 1}}
|
||||||
|
snapshot_count = 0
|
||||||
|
def collect_memory_snapshot():
|
||||||
|
global snapshot_count
|
||||||
|
snapshot_count += 1
|
||||||
|
return snapshot
|
||||||
|
def memory_reclaim_scan_diagnostics(): return {"ok": True, "blocked": False}
|
||||||
|
def job_paths(job_id): return {"state": "/tmp/fixture-state"}
|
||||||
|
written = []
|
||||||
|
def write_json_atomic(path, payload): written.append(payload)
|
||||||
|
def unexpected_fork(): raise RuntimeError("zero-candidate target-met run must not fork")
|
||||||
|
os.fork = unexpected_fork
|
||||||
|
plan = plan_payload("2026-07-21T00:00:00Z", {}, [], [], [])
|
||||||
|
run = start_memory_pressure_job("2026-07-21T00:00:00Z", [])
|
||||||
|
print(json.dumps({"plan": plan, "run": run, "written": written, "snapshotCount": snapshot_count}))
|
||||||
|
`);
|
||||||
|
const plan = result.plan as Record<string, any>;
|
||||||
|
const run = result.run as Record<string, any>;
|
||||||
|
assert.equal(plan.summary.targetMet, true);
|
||||||
|
assert.equal(plan.summary.targetGapBytes, 0);
|
||||||
|
assert.equal(plan.summary.outcome, "memory-target-met");
|
||||||
|
assert.equal(plan.summary.candidateCount, 0);
|
||||||
|
assert.equal(run.ok, true);
|
||||||
|
assert.equal(run.status, "succeeded");
|
||||||
|
assert.equal(run.outcome, "memory-target-met");
|
||||||
|
assert.equal(run.summary.targetMet, true);
|
||||||
|
assert.equal(run.summary.targetGapBytes, 0);
|
||||||
|
assert.equal(run.summary.outcome, "memory-target-met");
|
||||||
|
assert.equal(run.mutation, false);
|
||||||
|
assert.deepEqual(run.results, []);
|
||||||
|
assert.equal(result.snapshotCount, 2);
|
||||||
|
assert.deepEqual(result.written, [run]);
|
||||||
|
|
||||||
|
const config = Bun.YAML.parse(readFileSync(rootPath("config/unidesk-cli.yaml"), "utf8")) as Record<string, any>;
|
||||||
|
const projected = projectRemoteGcResult(plan, config.gc.remote.targets.NC01, {
|
||||||
|
action: "plan",
|
||||||
|
memoryPressureOnly: true,
|
||||||
|
full: false,
|
||||||
|
}) as Record<string, any>;
|
||||||
|
assert.equal(projected.runEligibility.allowed, true);
|
||||||
|
assert.equal(projected.runEligibility.allCandidatesEvaluated, true);
|
||||||
|
assert.deepEqual(projected.runEligibility.reasons, []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("memory-pressure zero-candidate run remains blocked below the target", () => {
|
||||||
|
const result = runRunnerPythonFixture([
|
||||||
|
"summarize_memory_candidates",
|
||||||
|
"apply_memory_target_summary",
|
||||||
|
"memory_pressure_selection",
|
||||||
|
"start_memory_pressure_job",
|
||||||
|
], String.raw`
|
||||||
|
PROVIDER_ID = "NC01"
|
||||||
|
OPTIONS = {"memoryPressureOnly": True, "memoryReclaimConfigId": "root-file-cache"}
|
||||||
|
target = 3221225472
|
||||||
|
MEMORY_CONFIG = {"cgroupReclaim": {"targetMemAvailableBytes": target}}
|
||||||
|
def safe_int(value, fallback=0):
|
||||||
|
try: return int(value)
|
||||||
|
except (TypeError, ValueError): return fallback
|
||||||
|
def fmt_bytes(value): return str(value)
|
||||||
|
def now_iso(): return "2026-07-21T00:00:01Z"
|
||||||
|
def status_command(job_id): return "gc remote NC01 status --job-id %s" % job_id
|
||||||
|
def collect_memory_snapshot(): return {"ok": True, "memory": {"availableBytes": target - 1}}
|
||||||
|
def memory_reclaim_scan_diagnostics(): return {"ok": True, "blocked": False}
|
||||||
|
def job_paths(job_id): return {"state": "/tmp/fixture-state"}
|
||||||
|
def write_json_atomic(path, payload): pass
|
||||||
|
run = start_memory_pressure_job("2026-07-21T00:00:00Z", [])
|
||||||
|
print(json.dumps(run))
|
||||||
|
`);
|
||||||
|
assert.equal(result.ok, true);
|
||||||
|
assert.equal(result.status, "blocked");
|
||||||
|
assert.equal(result.outcome, "no-memory-reclaim-candidates");
|
||||||
|
assert.equal((result.summary as Record<string, unknown>).targetMet, false);
|
||||||
|
assert.equal((result.summary as Record<string, unknown>).targetGapBytes, 1);
|
||||||
|
});
|
||||||
|
|
||||||
test("default memory-pressure plan projects an actionable YAML-bounded preview below stdout limit", () => {
|
test("default memory-pressure plan projects an actionable YAML-bounded preview below stdout limit", () => {
|
||||||
const processCandidate = (index: number): Record<string, unknown> => ({
|
const processCandidate = (index: number): Record<string, unknown> => ({
|
||||||
id: `browser-orphan-process-tree:${1000 + index}:12345`,
|
id: `browser-orphan-process-tree:${1000 + index}:12345`,
|
||||||
@@ -289,8 +654,8 @@ test("default memory-pressure plan projects an actionable YAML-bounded preview b
|
|||||||
assert.equal(projected.runEligibility.allCandidatesEvaluated, true);
|
assert.equal(projected.runEligibility.allCandidatesEvaluated, true);
|
||||||
assert.equal(projected.candidatePreview.limit, remoteTarget.memoryPressure.planPreviewLimit);
|
assert.equal(projected.candidatePreview.limit, remoteTarget.memoryPressure.planPreviewLimit);
|
||||||
assert.equal(projected.candidatePreview.candidates.length, Math.min(candidates.length, remoteTarget.memoryPressure.planPreviewLimit));
|
assert.equal(projected.candidatePreview.candidates.length, Math.min(candidates.length, remoteTarget.memoryPressure.planPreviewLimit));
|
||||||
assert.equal(projected.candidatePreview.candidates[2].identityClosure.identitiesComplete, true);
|
assert.equal(projected.candidatePreview.omittedCount, candidates.length - projected.candidatePreview.candidates.length);
|
||||||
assert.equal(projected.candidatePreview.candidates[2].processIdentities, undefined);
|
assert.ok(projected.candidatePreview.candidates.every((candidate: Record<string, unknown>) => candidate.processIdentities === undefined));
|
||||||
assert.equal(projected.policy.runCommand, "bun scripts/cli.ts gc remote NC01 run --confirm --memory-pressure-only");
|
assert.equal(projected.policy.runCommand, "bun scripts/cli.ts gc remote NC01 run --confirm --memory-pressure-only");
|
||||||
assert.equal(projected.valuesRedacted, true);
|
assert.equal(projected.valuesRedacted, true);
|
||||||
const renderedEnvelope = `${JSON.stringify({
|
const renderedEnvelope = `${JSON.stringify({
|
||||||
|
|||||||
@@ -1505,6 +1505,40 @@ def summarize_memory_candidates(candidates, returned):
|
|||||||
"byKind": by_kind,
|
"byKind": by_kind,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def apply_memory_target_summary(summary, memory_snapshot):
|
||||||
|
target_available = safe_int((MEMORY_CONFIG.get("cgroupReclaim") or {}).get("targetMemAvailableBytes"))
|
||||||
|
snapshot_memory = memory_snapshot.get("memory") if isinstance(memory_snapshot, dict) else None
|
||||||
|
available = safe_int(snapshot_memory.get("availableBytes")) if isinstance(snapshot_memory, dict) else 0
|
||||||
|
target_met = target_available > 0 and available >= target_available
|
||||||
|
summary.update({
|
||||||
|
"targetMemAvailableBytes": target_available,
|
||||||
|
"targetMet": target_met,
|
||||||
|
"targetGapBytes": max(0, target_available - available) if target_available > 0 else None,
|
||||||
|
})
|
||||||
|
if target_met:
|
||||||
|
summary["outcome"] = "memory-target-met"
|
||||||
|
return summary
|
||||||
|
|
||||||
|
|
||||||
|
def memory_pressure_selection():
|
||||||
|
config_id = OPTIONS.get("memoryReclaimConfigId")
|
||||||
|
if not isinstance(config_id, str) or not config_id:
|
||||||
|
return {"mode": "all-eligible", "configId": None}
|
||||||
|
return {
|
||||||
|
"mode": "yaml-cgroup-reclaim-config-id",
|
||||||
|
"configId": config_id,
|
||||||
|
"configSource": "config/unidesk-cli.yaml#gc.remote.targets.%s.memoryPressure.cgroupReclaim.targets[].id" % PROVIDER_ID,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def memory_pressure_option_suffix():
|
||||||
|
selection = memory_pressure_selection()
|
||||||
|
suffix = " --memory-pressure-only"
|
||||||
|
if selection.get("configId"):
|
||||||
|
suffix += " --memory-reclaim-config-id %s" % selection.get("configId")
|
||||||
|
return suffix
|
||||||
|
|
||||||
def assert_tmp_candidate(path):
|
def assert_tmp_candidate(path):
|
||||||
resolved = os.path.abspath(path)
|
resolved = os.path.abspath(path)
|
||||||
if not resolved.startswith("/tmp/"):
|
if not resolved.startswith("/tmp/"):
|
||||||
@@ -1674,6 +1708,10 @@ def returned_results(results):
|
|||||||
"hostMemAvailableDeltaBytes",
|
"hostMemAvailableDeltaBytes",
|
||||||
"kernelWriteAttempted",
|
"kernelWriteAttempted",
|
||||||
"kernelWriteError",
|
"kernelWriteError",
|
||||||
|
"rootFileReclaimEvidenceBefore",
|
||||||
|
"rootFileReclaimEvidenceAfter",
|
||||||
|
"targetGapBeforeBytes",
|
||||||
|
"targetGapAfterBytes",
|
||||||
"processCount",
|
"processCount",
|
||||||
"termSignalCount",
|
"termSignalCount",
|
||||||
"killSignalCount",
|
"killSignalCount",
|
||||||
@@ -1698,7 +1736,8 @@ def returned_results(results):
|
|||||||
def plan_payload(observed_at, preflight, protected, candidates, visible):
|
def plan_payload(observed_at, preflight, protected, candidates, visible):
|
||||||
if bool(OPTIONS.get("memoryPressureOnly")):
|
if bool(OPTIONS.get("memoryPressureOnly")):
|
||||||
scan = memory_reclaim_scan_diagnostics()
|
scan = memory_reclaim_scan_diagnostics()
|
||||||
summary = summarize_memory_candidates(candidates, visible)
|
memory_before = collect_memory_snapshot()
|
||||||
|
summary = apply_memory_target_summary(summarize_memory_candidates(candidates, visible), memory_before)
|
||||||
if scan.get("blocked"):
|
if scan.get("blocked"):
|
||||||
summary["outcome"] = "memory-reclaim-scan-blocked"
|
summary["outcome"] = "memory-reclaim-scan-blocked"
|
||||||
return {
|
return {
|
||||||
@@ -1710,8 +1749,9 @@ def plan_payload(observed_at, preflight, protected, candidates, visible):
|
|||||||
"mutation": False,
|
"mutation": False,
|
||||||
"observedAt": observed_at,
|
"observedAt": observed_at,
|
||||||
"options": OPTIONS,
|
"options": OPTIONS,
|
||||||
"memoryBefore": collect_memory_snapshot(),
|
"memoryBefore": memory_before,
|
||||||
"summary": summary,
|
"summary": summary,
|
||||||
|
"selection": memory_pressure_selection(),
|
||||||
"candidateScan": scan,
|
"candidateScan": scan,
|
||||||
"candidates": visible,
|
"candidates": visible,
|
||||||
"protected": {
|
"protected": {
|
||||||
@@ -1721,8 +1761,8 @@ def plan_payload(observed_at, preflight, protected, candidates, visible):
|
|||||||
},
|
},
|
||||||
"policy": {
|
"policy": {
|
||||||
"requiresRunConfirm": True,
|
"requiresRunConfirm": True,
|
||||||
"planCommand": "bun scripts/cli.ts gc remote %s plan --memory-pressure-only" % PROVIDER_ID,
|
"planCommand": "bun scripts/cli.ts gc remote %s plan%s" % (PROVIDER_ID, memory_pressure_option_suffix()),
|
||||||
"runCommand": "bun scripts/cli.ts gc remote %s run --confirm --memory-pressure-only" % PROVIDER_ID,
|
"runCommand": "bun scripts/cli.ts gc remote %s run --confirm%s" % (PROVIDER_ID, memory_pressure_option_suffix()),
|
||||||
"candidateKinds": ["cgroup-memory-reclaim", "browser-orphan-process-tree", "stale-web-observer-process-tree"],
|
"candidateKinds": ["cgroup-memory-reclaim", "browser-orphan-process-tree", "stale-web-observer-process-tree"],
|
||||||
"neverTouches": ["disk candidates", "apt cache", "journal", "tmp", "container images", "active observer process trees", "workload lifecycle"],
|
"neverTouches": ["disk candidates", "apt cache", "journal", "tmp", "container images", "active observer process trees", "workload lifecycle"],
|
||||||
"termination": "SIGTERM, bounded wait, identity reclose, SIGKILL only same-tree residual",
|
"termination": "SIGTERM, bounded wait, identity reclose, SIGKILL only same-tree residual",
|
||||||
@@ -2591,6 +2631,10 @@ def finish_memory_pressure_job(job_id, candidates, observed_at, worker_pid, work
|
|||||||
"lateAllowlistedIdentityCount": execution.get("lateAllowlistedIdentityCount"),
|
"lateAllowlistedIdentityCount": execution.get("lateAllowlistedIdentityCount"),
|
||||||
"memoryBefore": execution.get("memoryBefore"),
|
"memoryBefore": execution.get("memoryBefore"),
|
||||||
"memoryAfter": execution.get("memoryAfter"),
|
"memoryAfter": execution.get("memoryAfter"),
|
||||||
|
"rootFileReclaimEvidenceBefore": execution.get("rootFileReclaimEvidenceBefore"),
|
||||||
|
"rootFileReclaimEvidenceAfter": execution.get("rootFileReclaimEvidenceAfter"),
|
||||||
|
"targetGapBeforeBytes": execution.get("targetGapBeforeBytes"),
|
||||||
|
"targetGapAfterBytes": execution.get("targetGapAfterBytes"),
|
||||||
})
|
})
|
||||||
results.append(item)
|
results.append(item)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
@@ -2613,11 +2657,11 @@ def finish_memory_pressure_job(job_id, candidates, observed_at, worker_pid, work
|
|||||||
})
|
})
|
||||||
target_available = safe_int((MEMORY_CONFIG.get("cgroupReclaim") or {}).get("targetMemAvailableBytes"))
|
target_available = safe_int((MEMORY_CONFIG.get("cgroupReclaim") or {}).get("targetMemAvailableBytes"))
|
||||||
after_available = safe_int((memory_after.get("memory") or {}).get("availableBytes"))
|
after_available = safe_int((memory_after.get("memory") or {}).get("availableBytes"))
|
||||||
target_met = target_available > 0 and after_available > target_available
|
target_met = target_available > 0 and after_available >= target_available
|
||||||
summary.update({
|
summary.update({
|
||||||
"targetMemAvailableBytes": target_available,
|
"targetMemAvailableBytes": target_available,
|
||||||
"targetMet": target_met,
|
"targetMet": target_met,
|
||||||
"targetGapBytes": max(0, target_available - after_available + 1) if target_available > 0 else None,
|
"targetGapBytes": max(0, target_available - after_available) if target_available > 0 else None,
|
||||||
"outcome": "memory-target-met" if target_met else "memory-target-not-met",
|
"outcome": "memory-target-met" if target_met else "memory-target-not-met",
|
||||||
})
|
})
|
||||||
payload = {
|
payload = {
|
||||||
@@ -2643,6 +2687,7 @@ def finish_memory_pressure_job(job_id, candidates, observed_at, worker_pid, work
|
|||||||
"memoryBefore": memory_before,
|
"memoryBefore": memory_before,
|
||||||
"memoryAfter": memory_after,
|
"memoryAfter": memory_after,
|
||||||
"summary": summary,
|
"summary": summary,
|
||||||
|
"selection": memory_pressure_selection(),
|
||||||
"results": returned,
|
"results": returned,
|
||||||
}
|
}
|
||||||
write_memory_job_state(paths, payload)
|
write_memory_job_state(paths, payload)
|
||||||
@@ -2654,7 +2699,8 @@ def start_memory_pressure_job(observed_at, candidates):
|
|||||||
os.getpid(),
|
os.getpid(),
|
||||||
)
|
)
|
||||||
paths = job_paths(job_id)
|
paths = job_paths(job_id)
|
||||||
summary = summarize_memory_candidates(candidates, candidates)
|
memory_before = collect_memory_snapshot()
|
||||||
|
summary = apply_memory_target_summary(summarize_memory_candidates(candidates, candidates), memory_before)
|
||||||
scan = memory_reclaim_scan_diagnostics()
|
scan = memory_reclaim_scan_diagnostics()
|
||||||
base = {
|
base = {
|
||||||
"ok": scan.get("ok") is True,
|
"ok": scan.get("ok") is True,
|
||||||
@@ -2669,6 +2715,8 @@ def start_memory_pressure_job(observed_at, candidates):
|
|||||||
"observedAt": observed_at,
|
"observedAt": observed_at,
|
||||||
"startedAt": observed_at,
|
"startedAt": observed_at,
|
||||||
"summary": summary,
|
"summary": summary,
|
||||||
|
"memoryBefore": memory_before,
|
||||||
|
"selection": memory_pressure_selection(),
|
||||||
"candidateScan": scan,
|
"candidateScan": scan,
|
||||||
}
|
}
|
||||||
if scan.get("blocked"):
|
if scan.get("blocked"):
|
||||||
@@ -2683,6 +2731,19 @@ def start_memory_pressure_job(observed_at, candidates):
|
|||||||
write_json_atomic(paths["state"], terminal)
|
write_json_atomic(paths["state"], terminal)
|
||||||
return terminal
|
return terminal
|
||||||
if not candidates:
|
if not candidates:
|
||||||
|
if summary.get("targetMet") is True:
|
||||||
|
terminal = dict(base)
|
||||||
|
terminal.update({
|
||||||
|
"ok": True,
|
||||||
|
"status": "succeeded",
|
||||||
|
"outcome": "memory-target-met",
|
||||||
|
"mutation": False,
|
||||||
|
"finishedAt": now_iso(),
|
||||||
|
"memoryAfter": memory_before,
|
||||||
|
"results": [],
|
||||||
|
})
|
||||||
|
write_json_atomic(paths["state"], terminal)
|
||||||
|
return terminal
|
||||||
terminal = dict(base)
|
terminal = dict(base)
|
||||||
terminal.update({
|
terminal.update({
|
||||||
"status": "blocked",
|
"status": "blocked",
|
||||||
|
|||||||
@@ -323,12 +323,98 @@ def read_cgroup_memory_stat(path):
|
|||||||
with open(os.path.join(path, "memory.stat"), "r", encoding="utf-8") as handle:
|
with open(os.path.join(path, "memory.stat"), "r", encoding="utf-8") as handle:
|
||||||
for line in handle:
|
for line in handle:
|
||||||
parts = line.split()
|
parts = line.split()
|
||||||
if len(parts) == 2 and parts[0] in set(["anon", "file", "inactive_anon", "inactive_file"]):
|
if len(parts) == 2 and parts[0] in set(["anon", "file", "inactive_anon", "inactive_file", "slab_reclaimable"]):
|
||||||
selected[parts[0]] = safe_int(parts[1])
|
selected[parts[0]] = safe_int(parts[1])
|
||||||
except OSError:
|
except OSError:
|
||||||
return {}
|
return {}
|
||||||
return selected
|
return selected
|
||||||
|
|
||||||
|
def read_host_file_reclaim_evidence():
|
||||||
|
selected = {}
|
||||||
|
try:
|
||||||
|
with open("/proc/meminfo", "r", encoding="utf-8") as handle:
|
||||||
|
for line in handle:
|
||||||
|
key, separator, raw = line.partition(":")
|
||||||
|
if separator and key in set(["Cached", "Buffers", "SReclaimable"]):
|
||||||
|
value = raw.strip().split()[0]
|
||||||
|
selected[key] = int(value) * 1024
|
||||||
|
except (OSError, ValueError, IndexError):
|
||||||
|
return {"ok": False, "reason": "host-file-reclaim-evidence-unavailable"}
|
||||||
|
if any(key not in selected for key in ["Cached", "Buffers", "SReclaimable"]):
|
||||||
|
return {"ok": False, "reason": "host-file-reclaim-evidence-incomplete"}
|
||||||
|
cached_and_buffers = selected["Cached"] + selected["Buffers"]
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"source": "/proc/meminfo",
|
||||||
|
"cachedBytes": selected["Cached"],
|
||||||
|
"buffersBytes": selected["Buffers"],
|
||||||
|
"slabReclaimableBytes": selected["SReclaimable"],
|
||||||
|
"cachedAndBuffersBytes": cached_and_buffers,
|
||||||
|
"reclaimableFileSlabUpperBoundBytes": cached_and_buffers + selected["SReclaimable"],
|
||||||
|
"estimateKind": "kernel-reclaimable-upper-bound",
|
||||||
|
}
|
||||||
|
|
||||||
|
def resolve_root_cgroup_identity():
|
||||||
|
cgroup_root = (MEMORY_CONFIG.get("attribution") or {}).get("cgroupRoot")
|
||||||
|
if not isinstance(cgroup_root, str) or not os.path.isabs(cgroup_root) or os.path.normpath(cgroup_root) != cgroup_root:
|
||||||
|
return {"ok": False, "reason": "root-cgroup-config-path-invalid"}
|
||||||
|
if os.path.realpath(cgroup_root) != cgroup_root or not os.path.isdir(cgroup_root) or os.path.islink(cgroup_root):
|
||||||
|
return {"ok": False, "reason": "root-cgroup-path-unavailable-or-redirected"}
|
||||||
|
matches = []
|
||||||
|
try:
|
||||||
|
with open("/proc/self/mountinfo", "r", encoding="utf-8") as handle:
|
||||||
|
for line in handle:
|
||||||
|
before, separator, after = line.strip().partition(" - ")
|
||||||
|
if not separator:
|
||||||
|
continue
|
||||||
|
left = before.split()
|
||||||
|
right = after.split()
|
||||||
|
if len(left) < 6 or len(right) < 3 or right[0] != "cgroup2" or left[4] != cgroup_root:
|
||||||
|
continue
|
||||||
|
matches.append({
|
||||||
|
"mountId": safe_int(left[0], None),
|
||||||
|
"parentMountId": safe_int(left[1], None),
|
||||||
|
"majorMinor": left[2],
|
||||||
|
"mountRoot": left[3],
|
||||||
|
"mountPoint": left[4],
|
||||||
|
"filesystemType": right[0],
|
||||||
|
})
|
||||||
|
except OSError:
|
||||||
|
return {"ok": False, "reason": "root-cgroup-mountinfo-unavailable"}
|
||||||
|
if len(matches) != 1:
|
||||||
|
return {"ok": False, "reason": "root-cgroup-v2-mount-not-unique"}
|
||||||
|
identity = matches[0]
|
||||||
|
if identity.get("mountRoot") != "/" or identity.get("mountId") is None or identity.get("parentMountId") is None:
|
||||||
|
return {"ok": False, "reason": "configured-cgroup-path-is-not-mount-root"}
|
||||||
|
try:
|
||||||
|
stat = os.stat(cgroup_root, follow_symlinks=False)
|
||||||
|
except OSError:
|
||||||
|
return {"ok": False, "reason": "root-cgroup-stat-unavailable"}
|
||||||
|
stat_major_minor = "%s:%s" % (os.major(stat.st_dev), os.minor(stat.st_dev))
|
||||||
|
if stat_major_minor != identity.get("majorMinor") or stat.st_ino <= 0:
|
||||||
|
return {"ok": False, "reason": "root-cgroup-mount-identity-mismatch"}
|
||||||
|
identity["inode"] = stat.st_ino
|
||||||
|
identity["path"] = cgroup_root
|
||||||
|
return {"ok": True, "path": cgroup_root, "identity": identity}
|
||||||
|
|
||||||
|
def collect_root_file_reclaim_evidence(path):
|
||||||
|
host = read_host_file_reclaim_evidence()
|
||||||
|
stat = read_cgroup_memory_stat(path)
|
||||||
|
if host.get("ok") is not True:
|
||||||
|
return {"ok": False, "reason": host.get("reason"), "host": host, "rootMemoryStat": stat}
|
||||||
|
if any(key not in stat for key in ["file", "inactive_file", "slab_reclaimable"]):
|
||||||
|
return {"ok": False, "reason": "root-memory-stat-file-evidence-incomplete", "host": host, "rootMemoryStat": stat}
|
||||||
|
if safe_int(stat.get("inactive_file")) > safe_int(stat.get("file")):
|
||||||
|
return {"ok": False, "reason": "root-memory-stat-file-evidence-inconsistent", "host": host, "rootMemoryStat": stat}
|
||||||
|
root_upper_bound = safe_int(stat.get("file")) + safe_int(stat.get("slab_reclaimable"))
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"host": host,
|
||||||
|
"rootMemoryStat": stat,
|
||||||
|
"rootReclaimableFileSlabUpperBoundBytes": root_upper_bound,
|
||||||
|
"estimateKind": "kernel-reclaimable-upper-bound",
|
||||||
|
}
|
||||||
|
|
||||||
def configured_cgroup_reclaim_targets():
|
def configured_cgroup_reclaim_targets():
|
||||||
global _MEMORY_RECLAIM_CONFIG_ERROR
|
global _MEMORY_RECLAIM_CONFIG_ERROR
|
||||||
policy = MEMORY_CONFIG.get("cgroupReclaim")
|
policy = MEMORY_CONFIG.get("cgroupReclaim")
|
||||||
@@ -361,18 +447,25 @@ def configured_cgroup_reclaim_targets():
|
|||||||
path = item.get("path")
|
path = item.get("path")
|
||||||
workload_selector = item.get("workloadSelector")
|
workload_selector = item.get("workloadSelector")
|
||||||
systemd_unit_selector = item.get("systemdUnitSelector")
|
systemd_unit_selector = item.get("systemdUnitSelector")
|
||||||
|
root_cgroup_selector = item.get("rootCgroupSelector")
|
||||||
reclaim_bytes = item.get("reclaimBytes")
|
reclaim_bytes = item.get("reclaimBytes")
|
||||||
swappiness = item.get("swappiness")
|
swappiness = item.get("swappiness")
|
||||||
minimum_inactive_anon = item.get("minimumInactiveAnonBytes", 0)
|
minimum_inactive_anon = item.get("minimumInactiveAnonBytes", 0)
|
||||||
|
minimum_inactive_file = item.get("minimumInactiveFileBytes", 0)
|
||||||
|
minimum_file_slab = item.get("minimumReclaimableFileSlabBytes", 0)
|
||||||
if not isinstance(target_id, str) or not re.match(r"^[A-Za-z0-9._-]{1,80}$", target_id) or target_id in seen_ids:
|
if not isinstance(target_id, str) or not re.match(r"^[A-Za-z0-9._-]{1,80}$", target_id) or target_id in seen_ids:
|
||||||
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-target-id-invalid:%s" % index
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-target-id-invalid:%s" % index
|
||||||
return []
|
return []
|
||||||
if path is not None:
|
if path is not None:
|
||||||
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-path-selector-unsupported:%s" % target_id
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-path-selector-unsupported:%s" % target_id
|
||||||
return []
|
return []
|
||||||
if (workload_selector is None) == (systemd_unit_selector is None):
|
if sum(selector is not None for selector in [workload_selector, systemd_unit_selector, root_cgroup_selector]) != 1:
|
||||||
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-target-identity-invalid:%s" % target_id
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-target-identity-invalid:%s" % target_id
|
||||||
return []
|
return []
|
||||||
|
if root_cgroup_selector is not None:
|
||||||
|
if not isinstance(root_cgroup_selector, dict) or root_cgroup_selector != {"scope": "root"}:
|
||||||
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-root-selector-invalid:%s" % target_id
|
||||||
|
return []
|
||||||
if systemd_unit_selector is not None:
|
if systemd_unit_selector is not None:
|
||||||
if not isinstance(systemd_unit_selector, dict) or set(systemd_unit_selector.keys()) != set(["unit"]):
|
if not isinstance(systemd_unit_selector, dict) or set(systemd_unit_selector.keys()) != set(["unit"]):
|
||||||
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-systemd-selector-invalid:%s" % target_id
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-systemd-selector-invalid:%s" % target_id
|
||||||
@@ -413,6 +506,20 @@ def configured_cgroup_reclaim_targets():
|
|||||||
if swappiness > 0 and minimum_inactive_anon < reclaim_bytes:
|
if swappiness > 0 and minimum_inactive_anon < reclaim_bytes:
|
||||||
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-minimum-inactive-anon-too-small:%s" % target_id
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-minimum-inactive-anon-too-small:%s" % target_id
|
||||||
return []
|
return []
|
||||||
|
if any(isinstance(value, bool) or not isinstance(value, int) or value < 0 for value in [minimum_inactive_file, minimum_file_slab]):
|
||||||
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-file-lower-bound-invalid:%s" % target_id
|
||||||
|
return []
|
||||||
|
if root_cgroup_selector is not None and (
|
||||||
|
swappiness != 0
|
||||||
|
or minimum_inactive_anon != 0
|
||||||
|
or minimum_inactive_file <= 0
|
||||||
|
or minimum_file_slab < reclaim_bytes
|
||||||
|
):
|
||||||
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-root-cgroup-file-reclaim-policy-invalid:%s" % target_id
|
||||||
|
return []
|
||||||
|
if root_cgroup_selector is None and (minimum_inactive_file != 0 or minimum_file_slab != 0):
|
||||||
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-file-lower-bound-requires-root-selector:%s" % target_id
|
||||||
|
return []
|
||||||
priority = item.get("priority", index + 1)
|
priority = item.get("priority", index + 1)
|
||||||
if isinstance(priority, bool) or not isinstance(priority, int) or priority < 1 or priority > 1000:
|
if isinstance(priority, bool) or not isinstance(priority, int) or priority < 1 or priority > 1000:
|
||||||
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-priority-invalid:%s" % target_id
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-priority-invalid:%s" % target_id
|
||||||
@@ -422,9 +529,12 @@ def configured_cgroup_reclaim_targets():
|
|||||||
"id": target_id,
|
"id": target_id,
|
||||||
"workloadSelector": workload_selector,
|
"workloadSelector": workload_selector,
|
||||||
"systemdUnitSelector": systemd_unit_selector,
|
"systemdUnitSelector": systemd_unit_selector,
|
||||||
|
"rootCgroupSelector": root_cgroup_selector,
|
||||||
"reclaimBytes": reclaim_bytes,
|
"reclaimBytes": reclaim_bytes,
|
||||||
"swappiness": swappiness,
|
"swappiness": swappiness,
|
||||||
"minimumInactiveAnonBytes": minimum_inactive_anon,
|
"minimumInactiveAnonBytes": minimum_inactive_anon,
|
||||||
|
"minimumInactiveFileBytes": minimum_inactive_file,
|
||||||
|
"minimumReclaimableFileSlabBytes": minimum_file_slab,
|
||||||
"priority": priority,
|
"priority": priority,
|
||||||
"targetMemAvailableBytes": target_available,
|
"targetMemAvailableBytes": target_available,
|
||||||
"minimumSwapFreeBytesAfterReclaim": swap_reserve,
|
"minimumSwapFreeBytesAfterReclaim": swap_reserve,
|
||||||
@@ -487,6 +597,17 @@ def cgroup_reclaim_runtime_inventory():
|
|||||||
return _CGROUP_RECLAIM_RUNTIME_INVENTORY
|
return _CGROUP_RECLAIM_RUNTIME_INVENTORY
|
||||||
|
|
||||||
def resolve_cgroup_reclaim_target(item):
|
def resolve_cgroup_reclaim_target(item):
|
||||||
|
root_selector = item.get("rootCgroupSelector")
|
||||||
|
if isinstance(root_selector, dict):
|
||||||
|
resolved = resolve_root_cgroup_identity()
|
||||||
|
if resolved.get("ok") is not True:
|
||||||
|
return {"ok": False, "reason": resolved.get("reason"), "matches": []}
|
||||||
|
return {"ok": True, "matches": [{
|
||||||
|
"path": resolved.get("path"),
|
||||||
|
"pod": None,
|
||||||
|
"systemdUnit": None,
|
||||||
|
"rootCgroup": resolved.get("identity"),
|
||||||
|
}]}
|
||||||
systemd_selector = item.get("systemdUnitSelector")
|
systemd_selector = item.get("systemdUnitSelector")
|
||||||
if isinstance(systemd_selector, dict):
|
if isinstance(systemd_selector, dict):
|
||||||
unit = systemd_selector.get("unit")
|
unit = systemd_selector.get("unit")
|
||||||
@@ -528,7 +649,7 @@ def resolve_cgroup_reclaim_target(item):
|
|||||||
"mainPid": main_pid,
|
"mainPid": main_pid,
|
||||||
"startTicks": safe_int(start_ticks),
|
"startTicks": safe_int(start_ticks),
|
||||||
}
|
}
|
||||||
return {"ok": True, "matches": [{"path": path, "pod": None, "systemdUnit": identity}]}
|
return {"ok": True, "matches": [{"path": path, "pod": None, "systemdUnit": identity, "rootCgroup": None}]}
|
||||||
selector = item.get("workloadSelector") or {}
|
selector = item.get("workloadSelector") or {}
|
||||||
labels = selector.get("matchLabels") or {}
|
labels = selector.get("matchLabels") or {}
|
||||||
label_selector = ",".join("%s=%s" % pair for pair in sorted(labels.items()))
|
label_selector = ",".join("%s=%s" % pair for pair in sorted(labels.items()))
|
||||||
@@ -554,13 +675,31 @@ def resolve_cgroup_reclaim_target(item):
|
|||||||
"path": resolved_path,
|
"path": resolved_path,
|
||||||
"pod": {"namespace": selector.get("namespace"), "name": name, "uid": uid, "labelSelector": label_selector},
|
"pod": {"namespace": selector.get("namespace"), "name": name, "uid": uid, "labelSelector": label_selector},
|
||||||
"systemdUnit": None,
|
"systemdUnit": None,
|
||||||
|
"rootCgroup": None,
|
||||||
})
|
})
|
||||||
return {"ok": True, "matches": matches}
|
return {"ok": True, "matches": matches}
|
||||||
|
|
||||||
def collect_cgroup_reclaim_candidates():
|
def requested_memory_reclaim_config_id():
|
||||||
|
options = globals().get("OPTIONS")
|
||||||
|
value = options.get("memoryReclaimConfigId") if isinstance(options, dict) else None
|
||||||
|
return value if isinstance(value, str) and re.match(r"^[A-Za-z0-9._-]{1,80}$", value) else None
|
||||||
|
|
||||||
|
|
||||||
|
def collect_cgroup_reclaim_candidates(selected_config_id=None):
|
||||||
global _MEMORY_RECLAIM_CONFIG_ERROR, _CGROUP_RECLAIM_DIAGNOSTICS
|
global _MEMORY_RECLAIM_CONFIG_ERROR, _CGROUP_RECLAIM_DIAGNOSTICS
|
||||||
targets = configured_cgroup_reclaim_targets()
|
all_targets = configured_cgroup_reclaim_targets()
|
||||||
diagnostics = {"configuredCount": len(targets), "eligibleCount": 0, "protected": []}
|
targets = all_targets
|
||||||
|
if selected_config_id is not None:
|
||||||
|
targets = [item for item in all_targets if item.get("id") == selected_config_id]
|
||||||
|
if not targets and _MEMORY_RECLAIM_CONFIG_ERROR is None:
|
||||||
|
_MEMORY_RECLAIM_CONFIG_ERROR = "yaml-cgroup-reclaim-config-id-not-found:%s" % selected_config_id
|
||||||
|
diagnostics = {
|
||||||
|
"configuredCount": len(targets),
|
||||||
|
"totalConfiguredCount": len(all_targets),
|
||||||
|
"selectedConfigId": selected_config_id,
|
||||||
|
"eligibleCount": 0,
|
||||||
|
"protected": [],
|
||||||
|
}
|
||||||
_CGROUP_RECLAIM_DIAGNOSTICS = diagnostics
|
_CGROUP_RECLAIM_DIAGNOSTICS = diagnostics
|
||||||
if _MEMORY_RECLAIM_CONFIG_ERROR or not targets:
|
if _MEMORY_RECLAIM_CONFIG_ERROR or not targets:
|
||||||
return []
|
return []
|
||||||
@@ -578,6 +717,7 @@ def collect_cgroup_reclaim_candidates():
|
|||||||
"id": item.get("id"),
|
"id": item.get("id"),
|
||||||
"workloadSelector": item.get("workloadSelector"),
|
"workloadSelector": item.get("workloadSelector"),
|
||||||
"systemdUnitSelector": item.get("systemdUnitSelector"),
|
"systemdUnitSelector": item.get("systemdUnitSelector"),
|
||||||
|
"rootCgroupSelector": item.get("rootCgroupSelector"),
|
||||||
"reason": resolution.get("reason"),
|
"reason": resolution.get("reason"),
|
||||||
"matchedPodCount": resolution.get("matchedPodCount"),
|
"matchedPodCount": resolution.get("matchedPodCount"),
|
||||||
})
|
})
|
||||||
@@ -586,13 +726,14 @@ def collect_cgroup_reclaim_candidates():
|
|||||||
path = resolved.get("path")
|
path = resolved.get("path")
|
||||||
pod = resolved.get("pod")
|
pod = resolved.get("pod")
|
||||||
systemd_unit = resolved.get("systemdUnit")
|
systemd_unit = resolved.get("systemdUnit")
|
||||||
|
root_cgroup = resolved.get("rootCgroup")
|
||||||
reclaim_path = os.path.join(path, "memory.reclaim")
|
reclaim_path = os.path.join(path, "memory.reclaim")
|
||||||
protected_reason = None
|
protected_reason = None
|
||||||
if os.path.realpath(path) != path or not os.path.isdir(path) or os.path.islink(path):
|
if os.path.realpath(path) != path or not os.path.isdir(path) or os.path.islink(path):
|
||||||
protected_reason = "cgroup-path-unavailable-or-redirected"
|
protected_reason = "cgroup-path-unavailable-or-redirected"
|
||||||
elif not os.path.exists(reclaim_path) or not os.access(reclaim_path, os.W_OK):
|
elif not os.path.exists(reclaim_path) or not os.access(reclaim_path, os.W_OK):
|
||||||
protected_reason = "memory-reclaim-unavailable"
|
protected_reason = "memory-reclaim-unavailable"
|
||||||
elif available > safe_int(item.get("targetMemAvailableBytes")):
|
elif available >= safe_int(item.get("targetMemAvailableBytes")):
|
||||||
protected_reason = "target-memavailable-already-met"
|
protected_reason = "target-memavailable-already-met"
|
||||||
elif safe_int(item.get("swappiness")) > 0 and swap_free - safe_int(item.get("reclaimBytes")) < safe_int(item.get("minimumSwapFreeBytesAfterReclaim")):
|
elif safe_int(item.get("swappiness")) > 0 and swap_free - safe_int(item.get("reclaimBytes")) < safe_int(item.get("minimumSwapFreeBytesAfterReclaim")):
|
||||||
protected_reason = "minimum-swap-free-reserve"
|
protected_reason = "minimum-swap-free-reserve"
|
||||||
@@ -601,17 +742,31 @@ def collect_cgroup_reclaim_candidates():
|
|||||||
inactive_anon = safe_int(memory_stat.get("inactive_anon"))
|
inactive_anon = safe_int(memory_stat.get("inactive_anon"))
|
||||||
if protected_reason is None and safe_int(item.get("swappiness")) > 0 and inactive_anon < safe_int(item.get("minimumInactiveAnonBytes")):
|
if protected_reason is None and safe_int(item.get("swappiness")) > 0 and inactive_anon < safe_int(item.get("minimumInactiveAnonBytes")):
|
||||||
protected_reason = "minimum-inactive-anon-not-met"
|
protected_reason = "minimum-inactive-anon-not-met"
|
||||||
|
root_file_evidence = None
|
||||||
|
if protected_reason is None and item.get("rootCgroupSelector") is not None:
|
||||||
|
root_file_evidence = collect_root_file_reclaim_evidence(path)
|
||||||
|
if root_file_evidence.get("ok") is not True:
|
||||||
|
protected_reason = root_file_evidence.get("reason")
|
||||||
|
elif safe_int((root_file_evidence.get("rootMemoryStat") or {}).get("inactive_file")) < safe_int(item.get("minimumInactiveFileBytes")):
|
||||||
|
protected_reason = "minimum-root-inactive-file-not-met"
|
||||||
|
elif safe_int(root_file_evidence.get("rootReclaimableFileSlabUpperBoundBytes")) < safe_int(item.get("minimumReclaimableFileSlabBytes")):
|
||||||
|
protected_reason = "minimum-root-file-slab-upper-bound-not-met"
|
||||||
|
elif safe_int((root_file_evidence.get("host") or {}).get("reclaimableFileSlabUpperBoundBytes")) < safe_int(item.get("minimumReclaimableFileSlabBytes")):
|
||||||
|
protected_reason = "minimum-host-file-slab-upper-bound-not-met"
|
||||||
if protected_reason is not None:
|
if protected_reason is not None:
|
||||||
diagnostics["protected"].append({
|
diagnostics["protected"].append({
|
||||||
"id": item.get("id"),
|
"id": item.get("id"),
|
||||||
"path": path,
|
"path": path,
|
||||||
"workloadSelector": item.get("workloadSelector"),
|
"workloadSelector": item.get("workloadSelector"),
|
||||||
"systemdUnitSelector": item.get("systemdUnitSelector"),
|
"systemdUnitSelector": item.get("systemdUnitSelector"),
|
||||||
|
"rootCgroupSelector": item.get("rootCgroupSelector"),
|
||||||
"resolvedPod": pod,
|
"resolvedPod": pod,
|
||||||
"resolvedSystemdUnit": systemd_unit,
|
"resolvedSystemdUnit": systemd_unit,
|
||||||
|
"resolvedRootCgroup": root_cgroup,
|
||||||
"reason": protected_reason,
|
"reason": protected_reason,
|
||||||
"memoryCurrentBytes": current,
|
"memoryCurrentBytes": current,
|
||||||
"inactiveAnonBytes": inactive_anon,
|
"inactiveAnonBytes": inactive_anon,
|
||||||
|
"rootFileReclaimEvidence": root_file_evidence,
|
||||||
})
|
})
|
||||||
continue
|
continue
|
||||||
reclaim_bytes = safe_int(item.get("reclaimBytes"))
|
reclaim_bytes = safe_int(item.get("reclaimBytes"))
|
||||||
@@ -629,17 +784,22 @@ def collect_cgroup_reclaim_candidates():
|
|||||||
"path": path,
|
"path": path,
|
||||||
"workloadSelector": item.get("workloadSelector"),
|
"workloadSelector": item.get("workloadSelector"),
|
||||||
"systemdUnitSelector": item.get("systemdUnitSelector"),
|
"systemdUnitSelector": item.get("systemdUnitSelector"),
|
||||||
|
"rootCgroupSelector": item.get("rootCgroupSelector"),
|
||||||
"resolvedPod": pod,
|
"resolvedPod": pod,
|
||||||
"resolvedSystemdUnit": systemd_unit,
|
"resolvedSystemdUnit": systemd_unit,
|
||||||
|
"resolvedRootCgroup": root_cgroup,
|
||||||
"priority": safe_int(item.get("priority")),
|
"priority": safe_int(item.get("priority")),
|
||||||
"reclaimBytes": reclaim_bytes,
|
"reclaimBytes": reclaim_bytes,
|
||||||
"swappiness": safe_int(item.get("swappiness")),
|
"swappiness": safe_int(item.get("swappiness")),
|
||||||
"minimumInactiveAnonBytes": safe_int(item.get("minimumInactiveAnonBytes")),
|
"minimumInactiveAnonBytes": safe_int(item.get("minimumInactiveAnonBytes")),
|
||||||
|
"minimumInactiveFileBytes": safe_int(item.get("minimumInactiveFileBytes")),
|
||||||
|
"minimumReclaimableFileSlabBytes": safe_int(item.get("minimumReclaimableFileSlabBytes")),
|
||||||
"targetMemAvailableBytes": safe_int(item.get("targetMemAvailableBytes")),
|
"targetMemAvailableBytes": safe_int(item.get("targetMemAvailableBytes")),
|
||||||
"minimumSwapFreeBytesAfterReclaim": safe_int(item.get("minimumSwapFreeBytesAfterReclaim")),
|
"minimumSwapFreeBytesAfterReclaim": safe_int(item.get("minimumSwapFreeBytesAfterReclaim")),
|
||||||
"memoryCurrentBytes": current,
|
"memoryCurrentBytes": current,
|
||||||
"memorySwapCurrentBytes": read_cgroup_integer(path, "memory.swap.current"),
|
"memorySwapCurrentBytes": read_cgroup_integer(path, "memory.swap.current"),
|
||||||
"memoryStat": memory_stat,
|
"memoryStat": memory_stat,
|
||||||
|
"rootFileReclaimEvidence": root_file_evidence,
|
||||||
"requestedMemoryReclaimBytes": reclaim_bytes,
|
"requestedMemoryReclaimBytes": reclaim_bytes,
|
||||||
"requestedMemoryReclaim": fmt_bytes(reclaim_bytes),
|
"requestedMemoryReclaim": fmt_bytes(reclaim_bytes),
|
||||||
"estimateKind": "kernel-request-upper-bound",
|
"estimateKind": "kernel-request-upper-bound",
|
||||||
@@ -647,7 +807,7 @@ def collect_cgroup_reclaim_candidates():
|
|||||||
"expectedMemoryRss": fmt_bytes(reclaim_bytes),
|
"expectedMemoryRss": fmt_bytes(reclaim_bytes),
|
||||||
"estimatedDiskBytes": 0,
|
"estimatedDiskBytes": 0,
|
||||||
"estimatedReclaimBytes": 0,
|
"estimatedReclaimBytes": 0,
|
||||||
"action": {"op": "cgroup-v2-memory-reclaim", "allowlist": "yaml-stable-cgroup-selector"},
|
"action": {"op": "cgroup-v2-memory-reclaim", "allowlist": "yaml-root-cgroup-file-cache" if root_cgroup else "yaml-stable-cgroup-selector"},
|
||||||
})
|
})
|
||||||
diagnostics["eligibleCount"] = len(candidates)
|
diagnostics["eligibleCount"] = len(candidates)
|
||||||
return candidates
|
return candidates
|
||||||
@@ -663,6 +823,9 @@ def collect_memory_reclaim_candidates():
|
|||||||
_PROCESS_PRESSURE_DIAGNOSTICS = {}
|
_PROCESS_PRESSURE_DIAGNOSTICS = {}
|
||||||
_CGROUP_RECLAIM_DIAGNOSTICS = {"configuredCount": 0, "eligibleCount": 0, "protected": []}
|
_CGROUP_RECLAIM_DIAGNOSTICS = {"configuredCount": 0, "eligibleCount": 0, "protected": []}
|
||||||
_OBSERVE_RUN_CACHE.clear()
|
_OBSERVE_RUN_CACHE.clear()
|
||||||
|
selected_config_id = requested_memory_reclaim_config_id()
|
||||||
|
if selected_config_id is not None:
|
||||||
|
return collect_cgroup_reclaim_candidates(selected_config_id)
|
||||||
required_keys = [
|
required_keys = [
|
||||||
"observeStateRoots", "minimumReadableObserveRoots", "observeRunDepth", "observeScanLimit", "staleRunMaxAgeHours",
|
"observeStateRoots", "minimumReadableObserveRoots", "observeRunDepth", "observeScanLimit", "staleRunMaxAgeHours",
|
||||||
"processScanLimit", "processStaleMinAgeSeconds", "maxMemoryReclaimCandidates",
|
"processScanLimit", "processStaleMinAgeSeconds", "maxMemoryReclaimCandidates",
|
||||||
@@ -869,7 +1032,11 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
configured = next((item for item in configured_cgroup_reclaim_targets() if item.get("id") == config_id), None)
|
configured = next((item for item in configured_cgroup_reclaim_targets() if item.get("id") == config_id), None)
|
||||||
if configured is None:
|
if configured is None:
|
||||||
raise RuntimeError("cgroup reclaim candidate is no longer present in YAML allowlist")
|
raise RuntimeError("cgroup reclaim candidate is no longer present in YAML allowlist")
|
||||||
for key in ["workloadSelector", "systemdUnitSelector", "reclaimBytes", "swappiness", "minimumInactiveAnonBytes", "targetMemAvailableBytes", "minimumSwapFreeBytesAfterReclaim", "priority"]:
|
for key in [
|
||||||
|
"workloadSelector", "systemdUnitSelector", "rootCgroupSelector", "reclaimBytes", "swappiness",
|
||||||
|
"minimumInactiveAnonBytes", "minimumInactiveFileBytes", "minimumReclaimableFileSlabBytes",
|
||||||
|
"targetMemAvailableBytes", "minimumSwapFreeBytesAfterReclaim", "priority",
|
||||||
|
]:
|
||||||
if configured.get(key) != candidate.get(key):
|
if configured.get(key) != candidate.get(key):
|
||||||
raise RuntimeError("cgroup reclaim candidate no longer matches YAML: %s" % key)
|
raise RuntimeError("cgroup reclaim candidate no longer matches YAML: %s" % key)
|
||||||
_CGROUP_RECLAIM_RUNTIME_INVENTORY = None
|
_CGROUP_RECLAIM_RUNTIME_INVENTORY = None
|
||||||
@@ -879,10 +1046,12 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
path = candidate.get("path")
|
path = candidate.get("path")
|
||||||
resolved_uid = ((candidate.get("resolvedPod") or {}).get("uid"))
|
resolved_uid = ((candidate.get("resolvedPod") or {}).get("uid"))
|
||||||
resolved_systemd_unit = candidate.get("resolvedSystemdUnit")
|
resolved_systemd_unit = candidate.get("resolvedSystemdUnit")
|
||||||
|
resolved_root_cgroup = candidate.get("resolvedRootCgroup")
|
||||||
current_match = next((item for item in resolution.get("matches") or []
|
current_match = next((item for item in resolution.get("matches") or []
|
||||||
if item.get("path") == path
|
if item.get("path") == path
|
||||||
and ((item.get("pod") or {}).get("uid")) == resolved_uid
|
and ((item.get("pod") or {}).get("uid")) == resolved_uid
|
||||||
and item.get("systemdUnit") == resolved_systemd_unit
|
and item.get("systemdUnit") == resolved_systemd_unit
|
||||||
|
and item.get("rootCgroup") == resolved_root_cgroup
|
||||||
), None)
|
), None)
|
||||||
if current_match is None:
|
if current_match is None:
|
||||||
raise RuntimeError("cgroup reclaim candidate identity changed before execution")
|
raise RuntimeError("cgroup reclaim candidate identity changed before execution")
|
||||||
@@ -895,7 +1064,14 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
before_available = safe_int((before.get("memory") or {}).get("availableBytes"), None)
|
before_available = safe_int((before.get("memory") or {}).get("availableBytes"), None)
|
||||||
if before_available is None:
|
if before_available is None:
|
||||||
raise RuntimeError("MemAvailable is unavailable before cgroup reclaim")
|
raise RuntimeError("MemAvailable is unavailable before cgroup reclaim")
|
||||||
if before_available > safe_int(configured.get("targetMemAvailableBytes")):
|
root_evidence_before = None
|
||||||
|
if configured.get("rootCgroupSelector") is not None:
|
||||||
|
if safe_int(configured.get("swappiness")) != 0 or path != (resolved_root_cgroup or {}).get("mountPoint"):
|
||||||
|
raise RuntimeError("root cgroup file reclaim identity or swappiness is invalid")
|
||||||
|
root_evidence_before = collect_root_file_reclaim_evidence(path)
|
||||||
|
if root_evidence_before.get("ok") is not True:
|
||||||
|
raise RuntimeError("root cgroup file reclaim evidence is unavailable: %s" % root_evidence_before.get("reason"))
|
||||||
|
if before_available >= safe_int(configured.get("targetMemAvailableBytes")):
|
||||||
return {
|
return {
|
||||||
"status": "succeeded",
|
"status": "succeeded",
|
||||||
"outcome": "target-memavailable-already-met",
|
"outcome": "target-memavailable-already-met",
|
||||||
@@ -905,6 +1081,10 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
"cgroupMemoryBeforeBytes": read_cgroup_integer(path, "memory.current"),
|
"cgroupMemoryBeforeBytes": read_cgroup_integer(path, "memory.current"),
|
||||||
"cgroupMemoryAfterBytes": read_cgroup_integer(path, "memory.current"),
|
"cgroupMemoryAfterBytes": read_cgroup_integer(path, "memory.current"),
|
||||||
"kernelWriteAttempted": False,
|
"kernelWriteAttempted": False,
|
||||||
|
"rootFileReclaimEvidenceBefore": root_evidence_before,
|
||||||
|
"rootFileReclaimEvidenceAfter": root_evidence_before,
|
||||||
|
"targetGapBeforeBytes": 0,
|
||||||
|
"targetGapAfterBytes": 0,
|
||||||
}
|
}
|
||||||
swap_free = safe_int((before.get("swap") or {}).get("freeBytes"), None)
|
swap_free = safe_int((before.get("swap") or {}).get("freeBytes"), None)
|
||||||
if safe_int(configured.get("swappiness")) > 0:
|
if safe_int(configured.get("swappiness")) > 0:
|
||||||
@@ -916,6 +1096,13 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
inactive_anon = safe_int(read_cgroup_memory_stat(path).get("inactive_anon"))
|
inactive_anon = safe_int(read_cgroup_memory_stat(path).get("inactive_anon"))
|
||||||
if inactive_anon < safe_int(configured.get("minimumInactiveAnonBytes")):
|
if inactive_anon < safe_int(configured.get("minimumInactiveAnonBytes")):
|
||||||
raise RuntimeError("minimum inactive anonymous memory is no longer available")
|
raise RuntimeError("minimum inactive anonymous memory is no longer available")
|
||||||
|
if configured.get("rootCgroupSelector") is not None:
|
||||||
|
if safe_int((root_evidence_before.get("rootMemoryStat") or {}).get("inactive_file")) < safe_int(configured.get("minimumInactiveFileBytes")):
|
||||||
|
raise RuntimeError("minimum root inactive file memory is no longer available")
|
||||||
|
if safe_int(root_evidence_before.get("rootReclaimableFileSlabUpperBoundBytes")) < safe_int(configured.get("minimumReclaimableFileSlabBytes")):
|
||||||
|
raise RuntimeError("minimum root file/slab reclaim upper bound is no longer available")
|
||||||
|
if safe_int((root_evidence_before.get("host") or {}).get("reclaimableFileSlabUpperBoundBytes")) < safe_int(configured.get("minimumReclaimableFileSlabBytes")):
|
||||||
|
raise RuntimeError("minimum host file/slab reclaim upper bound is no longer available")
|
||||||
cgroup_before = read_cgroup_integer(path, "memory.current")
|
cgroup_before = read_cgroup_integer(path, "memory.current")
|
||||||
write_error = None
|
write_error = None
|
||||||
try:
|
try:
|
||||||
@@ -927,6 +1114,7 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
after = collect_memory_snapshot()
|
after = collect_memory_snapshot()
|
||||||
after_available = safe_int((after.get("memory") or {}).get("availableBytes"), None)
|
after_available = safe_int((after.get("memory") or {}).get("availableBytes"), None)
|
||||||
cgroup_after = read_cgroup_integer(path, "memory.current")
|
cgroup_after = read_cgroup_integer(path, "memory.current")
|
||||||
|
root_evidence_after = collect_root_file_reclaim_evidence(path) if configured.get("rootCgroupSelector") is not None else None
|
||||||
cgroup_delta = max(0, cgroup_before - cgroup_after) if cgroup_before is not None and cgroup_after is not None else 0
|
cgroup_delta = max(0, cgroup_before - cgroup_after) if cgroup_before is not None and cgroup_after is not None else 0
|
||||||
host_delta = max(0, after_available - before_available) if after_available is not None else 0
|
host_delta = max(0, after_available - before_available) if after_available is not None else 0
|
||||||
succeeded = write_error is None or cgroup_delta > 0 or host_delta > 0
|
succeeded = write_error is None or cgroup_delta > 0 or host_delta > 0
|
||||||
@@ -949,6 +1137,10 @@ def execute_cgroup_reclaim_candidate(candidate):
|
|||||||
"kernelWriteError": write_error,
|
"kernelWriteError": write_error,
|
||||||
"memoryBefore": before,
|
"memoryBefore": before,
|
||||||
"memoryAfter": after,
|
"memoryAfter": after,
|
||||||
|
"rootFileReclaimEvidenceBefore": root_evidence_before,
|
||||||
|
"rootFileReclaimEvidenceAfter": root_evidence_after,
|
||||||
|
"targetGapBeforeBytes": max(0, safe_int(configured.get("targetMemAvailableBytes")) - before_available),
|
||||||
|
"targetGapAfterBytes": max(0, safe_int(configured.get("targetMemAvailableBytes")) - after_available) if after_available is not None else None,
|
||||||
}
|
}
|
||||||
|
|
||||||
def execute_memory_process_candidate(candidate):
|
def execute_memory_process_candidate(candidate):
|
||||||
|
|||||||
@@ -38,6 +38,7 @@ interface RemoteGcOptions {
|
|||||||
historyLimit: number;
|
historyLimit: number;
|
||||||
saveSnapshot: boolean;
|
saveSnapshot: boolean;
|
||||||
memoryPressureOnly: boolean;
|
memoryPressureOnly: boolean;
|
||||||
|
memoryReclaimConfigId?: string;
|
||||||
buildCacheOnly: boolean;
|
buildCacheOnly: boolean;
|
||||||
hostDockerOnly: boolean;
|
hostDockerOnly: boolean;
|
||||||
}
|
}
|
||||||
@@ -71,6 +72,7 @@ const DEFAULT_REMOTE_OPTIONS: RemoteGcOptions = {
|
|||||||
historyLimit: 12,
|
historyLimit: 12,
|
||||||
saveSnapshot: true,
|
saveSnapshot: true,
|
||||||
memoryPressureOnly: false,
|
memoryPressureOnly: false,
|
||||||
|
memoryReclaimConfigId: undefined,
|
||||||
buildCacheOnly: false,
|
buildCacheOnly: false,
|
||||||
hostDockerOnly: false,
|
hostDockerOnly: false,
|
||||||
};
|
};
|
||||||
@@ -182,6 +184,14 @@ export async function runRemoteGcCommand(config: UniDeskConfig, providerId: stri
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
const options = parseRemoteGcOptions(args);
|
const options = parseRemoteGcOptions(args);
|
||||||
|
if (options.memoryReclaimConfigId !== undefined && !options.memoryPressureOnly) {
|
||||||
|
return {
|
||||||
|
ok: false,
|
||||||
|
error: "gc-remote-memory-reclaim-config-id-requires-memory-pressure-only",
|
||||||
|
configId: options.memoryReclaimConfigId,
|
||||||
|
requiredFlag: "--memory-pressure-only",
|
||||||
|
};
|
||||||
|
}
|
||||||
if (options.memoryPressureOnly && subaction !== "plan" && subaction !== "run") {
|
if (options.memoryPressureOnly && subaction !== "plan" && subaction !== "run") {
|
||||||
return {
|
return {
|
||||||
ok: false,
|
ok: false,
|
||||||
@@ -209,8 +219,8 @@ export async function runRemoteGcCommand(config: UniDeskConfig, providerId: stri
|
|||||||
dryRun: true,
|
dryRun: true,
|
||||||
mutation: false,
|
mutation: false,
|
||||||
requiredFlag: "--confirm",
|
requiredFlag: "--confirm",
|
||||||
planCommand: `bun scripts/cli.ts gc remote ${providerId} plan${options.memoryPressureOnly ? " --memory-pressure-only" : registryFlag}`,
|
planCommand: `bun scripts/cli.ts gc remote ${providerId} plan${options.memoryPressureOnly ? memoryPressureOptionSuffix(options) : registryFlag}`,
|
||||||
runCommand: `bun scripts/cli.ts gc remote ${providerId} run --confirm${options.memoryPressureOnly ? " --memory-pressure-only" : registryFlag}`,
|
runCommand: `bun scripts/cli.ts gc remote ${providerId} run --confirm${options.memoryPressureOnly ? memoryPressureOptionSuffix(options) : registryFlag}`,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
return await runRemoteGc(config, providerId, "run", options);
|
return await runRemoteGc(config, providerId, "run", options);
|
||||||
@@ -317,6 +327,10 @@ function parseRemoteGcOptions(args: string[]): RemoteGcOptions {
|
|||||||
options.saveSnapshot = false;
|
options.saveSnapshot = false;
|
||||||
} else if (arg === "--memory-pressure-only") {
|
} else if (arg === "--memory-pressure-only") {
|
||||||
options.memoryPressureOnly = true;
|
options.memoryPressureOnly = true;
|
||||||
|
} else if (arg === "--memory-reclaim-config-id") {
|
||||||
|
const value = args[++index];
|
||||||
|
if (!value || !/^[A-Za-z0-9._-]{1,80}$/u.test(value)) throw new Error("--memory-reclaim-config-id must be a YAML cgroupReclaim target id");
|
||||||
|
options.memoryReclaimConfigId = value;
|
||||||
} else if (arg === "--build-cache-only") {
|
} else if (arg === "--build-cache-only") {
|
||||||
options.buildCacheOnly = true;
|
options.buildCacheOnly = true;
|
||||||
} else if (arg === "--host-docker-only") {
|
} else if (arg === "--host-docker-only") {
|
||||||
@@ -328,6 +342,13 @@ function parseRemoteGcOptions(args: string[]): RemoteGcOptions {
|
|||||||
return options;
|
return options;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function memoryPressureOptionSuffix(options: RemoteGcOptions): string {
|
||||||
|
const selection = options.memoryReclaimConfigId === undefined
|
||||||
|
? ""
|
||||||
|
: ` --memory-reclaim-config-id ${options.memoryReclaimConfigId}`;
|
||||||
|
return ` --memory-pressure-only${selection}`;
|
||||||
|
}
|
||||||
|
|
||||||
function parseNonNegativeNumber(name: string, raw: string | undefined): number {
|
function parseNonNegativeNumber(name: string, raw: string | undefined): number {
|
||||||
const value = Number(raw);
|
const value = Number(raw);
|
||||||
if (!Number.isFinite(value) || value < 0) throw new Error(`${name} must be a non-negative number`);
|
if (!Number.isFinite(value) || value < 0) throw new Error(`${name} must be a non-negative number`);
|
||||||
@@ -453,6 +474,45 @@ function compactResolvedSystemdUnit(value: unknown): Record<string, unknown> | u
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function compactRootCgroupSelector(value: unknown): string | undefined {
|
||||||
|
const selector = recordOrEmpty(value);
|
||||||
|
return selector.scope === "root" && Object.keys(selector).length === 1 ? "root" : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function compactResolvedRootCgroup(value: unknown): Record<string, unknown> | undefined {
|
||||||
|
const identity = recordOrEmpty(value);
|
||||||
|
if (identity.filesystemType !== "cgroup2"
|
||||||
|
|| identity.mountRoot !== "/"
|
||||||
|
|| typeof identity.mountPoint !== "string"
|
||||||
|
|| typeof identity.majorMinor !== "string"
|
||||||
|
|| positiveInteger(identity.mountId) === null
|
||||||
|
|| positiveInteger(identity.parentMountId) === null
|
||||||
|
|| positiveInteger(identity.inode) === null) return undefined;
|
||||||
|
return {
|
||||||
|
filesystemType: identity.filesystemType,
|
||||||
|
mountRoot: identity.mountRoot,
|
||||||
|
mountPoint: identity.mountPoint,
|
||||||
|
majorMinor: identity.majorMinor,
|
||||||
|
mountId: identity.mountId,
|
||||||
|
parentMountId: identity.parentMountId,
|
||||||
|
inode: identity.inode,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function compactRootFileReclaimEvidence(value: unknown): Record<string, unknown> | undefined {
|
||||||
|
const evidence = recordOrEmpty(value);
|
||||||
|
const host = recordOrEmpty(evidence.host);
|
||||||
|
const rootMemoryStat = recordOrEmpty(evidence.rootMemoryStat);
|
||||||
|
if (evidence.ok !== true || host.source !== "/proc/meminfo") return undefined;
|
||||||
|
return {
|
||||||
|
source: host.source,
|
||||||
|
estimateKind: evidence.estimateKind,
|
||||||
|
hostFileSlabUpperBoundBytes: host.reclaimableFileSlabUpperBoundBytes,
|
||||||
|
rootFileSlabUpperBoundBytes: evidence.rootReclaimableFileSlabUpperBoundBytes,
|
||||||
|
rootInactiveFileBytes: rootMemoryStat.inactive_file,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
function compactMemoryPressureCandidate(value: unknown): Record<string, unknown> {
|
function compactMemoryPressureCandidate(value: unknown): Record<string, unknown> {
|
||||||
const candidate = recordOrEmpty(value);
|
const candidate = recordOrEmpty(value);
|
||||||
const action = recordOrEmpty(candidate.action);
|
const action = recordOrEmpty(candidate.action);
|
||||||
@@ -476,14 +536,22 @@ function compactMemoryPressureCandidate(value: unknown): Record<string, unknown>
|
|||||||
resolvedPod: compactResolvedPod(candidate.resolvedPod),
|
resolvedPod: compactResolvedPod(candidate.resolvedPod),
|
||||||
systemdUnitSelector: compactSystemdUnitSelector(candidate.systemdUnitSelector),
|
systemdUnitSelector: compactSystemdUnitSelector(candidate.systemdUnitSelector),
|
||||||
resolvedSystemdUnit: compactResolvedSystemdUnit(candidate.resolvedSystemdUnit),
|
resolvedSystemdUnit: compactResolvedSystemdUnit(candidate.resolvedSystemdUnit),
|
||||||
|
rootCgroupSelector: compactRootCgroupSelector(candidate.rootCgroupSelector),
|
||||||
|
resolvedRootCgroup: compactResolvedRootCgroup(candidate.resolvedRootCgroup),
|
||||||
requestedMemoryReclaimBytes: candidate.requestedMemoryReclaimBytes,
|
requestedMemoryReclaimBytes: candidate.requestedMemoryReclaimBytes,
|
||||||
requestedMemoryReclaim: candidate.requestedMemoryReclaim,
|
requestedMemoryReclaim: candidate.requestedMemoryReclaim,
|
||||||
estimateKind: candidate.estimateKind,
|
estimateKind: candidate.estimateKind,
|
||||||
swappiness: candidate.swappiness,
|
swappiness: candidate.swappiness,
|
||||||
minimumInactiveAnonBytes: candidate.minimumInactiveAnonBytes,
|
minimumInactiveAnonBytes: candidate.minimumInactiveAnonBytes,
|
||||||
|
minimumInactiveFileBytes: candidate.minimumInactiveFileBytes,
|
||||||
|
minimumReclaimableFileSlabBytes: candidate.minimumReclaimableFileSlabBytes,
|
||||||
memoryCurrentBytes: candidate.memoryCurrentBytes,
|
memoryCurrentBytes: candidate.memoryCurrentBytes,
|
||||||
memorySwapCurrentBytes: candidate.memorySwapCurrentBytes,
|
memorySwapCurrentBytes: candidate.memorySwapCurrentBytes,
|
||||||
inactiveAnonBytes: memoryStat.inactive_anon,
|
inactiveAnonBytes: memoryStat.inactive_anon,
|
||||||
|
inactiveFileBytes: memoryStat.inactive_file,
|
||||||
|
fileBytes: memoryStat.file,
|
||||||
|
slabReclaimableBytes: memoryStat.slab_reclaimable,
|
||||||
|
rootFileReclaimEvidence: compactRootFileReclaimEvidence(candidate.rootFileReclaimEvidence),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
const identities = Array.isArray(candidate.processIdentities) ? candidate.processIdentities.map(recordOrEmpty) : [];
|
const identities = Array.isArray(candidate.processIdentities) ? candidate.processIdentities.map(recordOrEmpty) : [];
|
||||||
@@ -524,6 +592,11 @@ function memoryPressureCandidateAllowed(value: unknown): boolean {
|
|||||||
const resolvedPod = recordOrEmpty(candidate.resolvedPod);
|
const resolvedPod = recordOrEmpty(candidate.resolvedPod);
|
||||||
const systemdUnitSelector = recordOrEmpty(candidate.systemdUnitSelector);
|
const systemdUnitSelector = recordOrEmpty(candidate.systemdUnitSelector);
|
||||||
const resolvedSystemdUnit = recordOrEmpty(candidate.resolvedSystemdUnit);
|
const resolvedSystemdUnit = recordOrEmpty(candidate.resolvedSystemdUnit);
|
||||||
|
const rootCgroupSelector = compactRootCgroupSelector(candidate.rootCgroupSelector);
|
||||||
|
const resolvedRootCgroup = compactResolvedRootCgroup(candidate.resolvedRootCgroup);
|
||||||
|
const reclaimBytes = positiveInteger(candidate.reclaimBytes);
|
||||||
|
const minimumInactiveFileBytes = positiveInteger(candidate.minimumInactiveFileBytes);
|
||||||
|
const minimumReclaimableFileSlabBytes = positiveInteger(candidate.minimumReclaimableFileSlabBytes);
|
||||||
const workloadIdentityClosed = typeof workloadSelector.namespace === "string" && typeof resolvedPod.uid === "string";
|
const workloadIdentityClosed = typeof workloadSelector.namespace === "string" && typeof resolvedPod.uid === "string";
|
||||||
const systemdIdentityClosed = typeof systemdUnitSelector.unit === "string"
|
const systemdIdentityClosed = typeof systemdUnitSelector.unit === "string"
|
||||||
&& resolvedSystemdUnit.unit === systemdUnitSelector.unit
|
&& resolvedSystemdUnit.unit === systemdUnitSelector.unit
|
||||||
@@ -535,14 +608,23 @@ function memoryPressureCandidateAllowed(value: unknown): boolean {
|
|||||||
&& candidate.path === `/sys/fs/cgroup${resolvedSystemdUnit.controlGroup}`
|
&& candidate.path === `/sys/fs/cgroup${resolvedSystemdUnit.controlGroup}`
|
||||||
&& positiveInteger(resolvedSystemdUnit.mainPid) !== null
|
&& positiveInteger(resolvedSystemdUnit.mainPid) !== null
|
||||||
&& positiveInteger(resolvedSystemdUnit.startTicks) !== null;
|
&& positiveInteger(resolvedSystemdUnit.startTicks) !== null;
|
||||||
|
const rootIdentityClosed = rootCgroupSelector === "root"
|
||||||
|
&& resolvedRootCgroup !== undefined
|
||||||
|
&& candidate.path === resolvedRootCgroup.mountPoint
|
||||||
|
&& swappiness === 0
|
||||||
|
&& reclaimBytes !== null
|
||||||
|
&& minimumInactiveFileBytes !== null
|
||||||
|
&& minimumReclaimableFileSlabBytes !== null
|
||||||
|
&& minimumReclaimableFileSlabBytes >= reclaimBytes;
|
||||||
return typeof candidate.configId === "string" && candidate.configId.length > 0
|
return typeof candidate.configId === "string" && candidate.configId.length > 0
|
||||||
&& typeof candidate.path === "string" && candidate.path.startsWith("/")
|
&& typeof candidate.path === "string" && candidate.path.startsWith("/")
|
||||||
&& positiveInteger(candidate.reclaimBytes) !== null
|
&& reclaimBytes !== null
|
||||||
&& swappiness !== null && swappiness >= 0 && swappiness <= 200
|
&& swappiness !== null && swappiness >= 0 && swappiness <= 200
|
||||||
&& (swappiness === 0 || positiveInteger(candidate.minimumInactiveAnonBytes) !== null)
|
&& (swappiness === 0 || positiveInteger(candidate.minimumInactiveAnonBytes) !== null)
|
||||||
&& action.op === "cgroup-v2-memory-reclaim"
|
&& action.op === "cgroup-v2-memory-reclaim"
|
||||||
&& action.allowlist === "yaml-stable-cgroup-selector"
|
&& (action.allowlist === "yaml-stable-cgroup-selector" || action.allowlist === "yaml-root-cgroup-file-cache")
|
||||||
&& workloadIdentityClosed !== systemdIdentityClosed;
|
&& [workloadIdentityClosed, systemdIdentityClosed, rootIdentityClosed].filter(Boolean).length === 1
|
||||||
|
&& (rootIdentityClosed === (action.allowlist === "yaml-root-cgroup-file-cache"));
|
||||||
}
|
}
|
||||||
if (candidate.kind !== "browser-orphan-process-tree" && candidate.kind !== "stale-web-observer-process-tree") return false;
|
if (candidate.kind !== "browser-orphan-process-tree" && candidate.kind !== "stale-web-observer-process-tree") return false;
|
||||||
const compact = compactMemoryPressureCandidate(candidate);
|
const compact = compactMemoryPressureCandidate(candidate);
|
||||||
@@ -586,6 +668,8 @@ function compactMemoryPressureCandidateScan(value: unknown): Record<string, unkn
|
|||||||
},
|
},
|
||||||
cgroupReclaim: {
|
cgroupReclaim: {
|
||||||
configuredCount: cgroupReclaim.configuredCount,
|
configuredCount: cgroupReclaim.configuredCount,
|
||||||
|
totalConfiguredCount: cgroupReclaim.totalConfiguredCount,
|
||||||
|
selectedConfigId: cgroupReclaim.selectedConfigId,
|
||||||
eligibleCount: cgroupReclaim.eligibleCount,
|
eligibleCount: cgroupReclaim.eligibleCount,
|
||||||
protectedCount: cgroupProtected.length,
|
protectedCount: cgroupProtected.length,
|
||||||
protected: cgroupProtected.map((item) => {
|
protected: cgroupProtected.map((item) => {
|
||||||
@@ -596,6 +680,9 @@ function compactMemoryPressureCandidateScan(value: unknown): Record<string, unkn
|
|||||||
matchedPodCount: protectedItem.matchedPodCount,
|
matchedPodCount: protectedItem.matchedPodCount,
|
||||||
memoryCurrentBytes: protectedItem.memoryCurrentBytes,
|
memoryCurrentBytes: protectedItem.memoryCurrentBytes,
|
||||||
inactiveAnonBytes: protectedItem.inactiveAnonBytes,
|
inactiveAnonBytes: protectedItem.inactiveAnonBytes,
|
||||||
|
rootCgroupSelector: compactRootCgroupSelector(protectedItem.rootCgroupSelector),
|
||||||
|
resolvedRootCgroup: compactResolvedRootCgroup(protectedItem.resolvedRootCgroup),
|
||||||
|
rootFileReclaimEvidence: compactRootFileReclaimEvidence(protectedItem.rootFileReclaimEvidence),
|
||||||
workloadSelector: compactWorkloadSelector(protectedItem.workloadSelector),
|
workloadSelector: compactWorkloadSelector(protectedItem.workloadSelector),
|
||||||
resolvedPod: compactResolvedPod(protectedItem.resolvedPod),
|
resolvedPod: compactResolvedPod(protectedItem.resolvedPod),
|
||||||
systemdUnitSelector: compactSystemdUnitSelector(protectedItem.systemdUnitSelector),
|
systemdUnitSelector: compactSystemdUnitSelector(protectedItem.systemdUnitSelector),
|
||||||
@@ -620,11 +707,12 @@ export function projectRemoteGcResult(
|
|||||||
const memoryPressure = recordOrEmpty(remoteTarget.memoryPressure);
|
const memoryPressure = recordOrEmpty(remoteTarget.memoryPressure);
|
||||||
const previewLimit = positiveInteger(memoryPressure.planPreviewLimit);
|
const previewLimit = positiveInteger(memoryPressure.planPreviewLimit);
|
||||||
const totalCandidateCount = positiveInteger(summary.candidateCount) ?? 0;
|
const totalCandidateCount = positiveInteger(summary.candidateCount) ?? 0;
|
||||||
const allCandidatesEvaluated = totalCandidateCount > 0 && totalCandidateCount === candidates.length;
|
const targetMet = summary.targetMet === true;
|
||||||
|
const allCandidatesEvaluated = totalCandidateCount === candidates.length;
|
||||||
const reasons: string[] = [];
|
const reasons: string[] = [];
|
||||||
if (payload.ok !== true) reasons.push("plan-not-ok");
|
if (payload.ok !== true) reasons.push("plan-not-ok");
|
||||||
if (scan.ok !== true || scan.blocked === true) reasons.push("candidate-scan-blocked");
|
if (scan.ok !== true || scan.blocked === true) reasons.push("candidate-scan-blocked");
|
||||||
if (totalCandidateCount === 0) reasons.push("no-candidates");
|
if (totalCandidateCount === 0 && !targetMet) reasons.push("no-candidates");
|
||||||
if (!allCandidatesEvaluated) reasons.push("candidate-page-incomplete");
|
if (!allCandidatesEvaluated) reasons.push("candidate-page-incomplete");
|
||||||
if (!candidates.every(memoryPressureCandidateAllowed)) reasons.push("unsupported-or-incomplete-candidate");
|
if (!candidates.every(memoryPressureCandidateAllowed)) reasons.push("unsupported-or-incomplete-candidate");
|
||||||
if (previewLimit === null) reasons.push("yaml-plan-preview-limit-invalid");
|
if (previewLimit === null) reasons.push("yaml-plan-preview-limit-invalid");
|
||||||
@@ -641,6 +729,7 @@ export function projectRemoteGcResult(
|
|||||||
observedAt: payload.observedAt,
|
observedAt: payload.observedAt,
|
||||||
memoryBefore: payload.memoryBefore,
|
memoryBefore: payload.memoryBefore,
|
||||||
summary: payload.summary,
|
summary: payload.summary,
|
||||||
|
selection: payload.selection,
|
||||||
candidateScan: compactScan,
|
candidateScan: compactScan,
|
||||||
runEligibility: {
|
runEligibility: {
|
||||||
allowed: reasons.length === 0,
|
allowed: reasons.length === 0,
|
||||||
|
|||||||
@@ -527,6 +527,8 @@ function gcHelp(): unknown {
|
|||||||
"bun scripts/cli.ts gc remote NC01 retention run --confirm --limit 500",
|
"bun scripts/cli.ts gc remote NC01 retention run --confirm --limit 500",
|
||||||
"bun scripts/cli.ts gc remote NC01 plan --memory-pressure-only",
|
"bun scripts/cli.ts gc remote NC01 plan --memory-pressure-only",
|
||||||
"bun scripts/cli.ts gc remote NC01 run --confirm --memory-pressure-only",
|
"bun scripts/cli.ts gc remote NC01 run --confirm --memory-pressure-only",
|
||||||
|
"bun scripts/cli.ts gc remote NC01 plan --memory-pressure-only --memory-reclaim-config-id <yaml-target-id>",
|
||||||
|
"bun scripts/cli.ts gc remote NC01 run --confirm --memory-pressure-only --memory-reclaim-config-id <yaml-target-id>",
|
||||||
"bun scripts/cli.ts gc remote NC01 status --job-id <job-id>",
|
"bun scripts/cli.ts gc remote NC01 status --job-id <job-id>",
|
||||||
"bun scripts/cli.ts gc remote G14 plan --target-use-percent 50 --include-hwlab-registry",
|
"bun scripts/cli.ts gc remote G14 plan --target-use-percent 50 --include-hwlab-registry",
|
||||||
"bun scripts/cli.ts gc remote G14 run --confirm",
|
"bun scripts/cli.ts gc remote G14 run --confirm",
|
||||||
@@ -558,6 +560,7 @@ function gcHelp(): unknown {
|
|||||||
"--registry-min-age-hours N": "remote registry only: keep all tags newer than N hours; default 48, minimum 0",
|
"--registry-min-age-hours N": "remote registry only: keep all tags newer than N hours; default 48, minimum 0",
|
||||||
"--target-use-percent N": "evaluate whether planned candidates can reduce root filesystem use to N%; reports required reclaim, projected use, shortfall and safe-stop decision",
|
"--target-use-percent N": "evaluate whether planned candidates can reduce root filesystem use to N%; reports required reclaim, projected use, shortfall and safe-stop decision",
|
||||||
"--job-id ID": "remote status only: inspect a long-running remote gc job",
|
"--job-id ID": "remote status only: inspect a long-running remote gc job",
|
||||||
|
"--memory-reclaim-config-id ID": "remote memory-pressure plan/run only: select exactly one owning YAML cgroupReclaim target id and exclude every other cgroup or process candidate",
|
||||||
"--limit N": "number of candidates returned and executed by run when --full is not set; default 50",
|
"--limit N": "number of candidates returned and executed by run when --full is not set; default 50",
|
||||||
"--result-limit N": "number of per-candidate run results returned when --full is not set; default 50",
|
"--result-limit N": "number of per-candidate run results returned when --full is not set; default 50",
|
||||||
"--full|--raw": "return and run against all candidates rather than the default bounded page",
|
"--full|--raw": "return and run against all candidates rather than the default bounded page",
|
||||||
|
|||||||
Reference in New Issue
Block a user