feat: 为 k3s worker 部署受控出网代理
This commit is contained in:
@@ -354,6 +354,7 @@ bun scripts/cli.ts hwlab nodes control-plane legacy-cicd --help
|
||||
- k3s drop-in 未变化且节点容量已收敛时必须 `mutation=false`;禁止无条件重启、重装、强写或通过 CI 自动覆盖同一节点配置。
|
||||
- WireGuard 配置一致时不得 restart;k3s API/容量暂时不可读时保持 unknown 并停止 mutation,禁止把控制通道抖动解释为配置漂移。
|
||||
- worker agent 只在 YAML 派生摘要变化或 service 未运行时安装/恢复,不得在每次集群 apply 中先卸载再重装。
|
||||
- worker 必须先由集群入口按 `hostProxyConfigRef` 部署并验证 proxy client,再启动 agent;containerd 通过该 proxy 直接访问上游 registry,禁止用公共 registry mirror 代替出网链路。
|
||||
- worker 安装使用短启动并由 control-plane Node Ready 轮询收口,禁止让 `trans`/SSH 长挂等待 systemd notify。
|
||||
- worker 加入集群不改变 PaC consumer 的 source commit authority,也不授权人工 PipelineRun、Argo sync 或业务 rollout。
|
||||
- 长命令用异步 job 或短轮询;不要长时间挂住 trans/ssh。
|
||||
|
||||
@@ -57,6 +57,13 @@ D601/D518 结果必须分表记录:`STATE`、Job/run、duration、`APK/NPM/GO/
|
||||
|
||||
## Egress Proxy 运行面修复入口
|
||||
|
||||
- 多主机 k3s worker 的 bootstrap proxy:
|
||||
- 集群 owning YAML 通过 `hostProxyConfigRef` 组合 `config/platform-infra/host-proxy.yaml`。
|
||||
- 部署和验收统一走 `platform-infra k3s-cluster apply|status`。
|
||||
- 集群入口经 Windows 宿主内网 SSH 分发静态客户端到 guest。
|
||||
- 启动 `k3s-agent` 前必须验证外网 probe。
|
||||
- containerd 直接通过 host proxy 访问上游 registry,不使用公共镜像站。
|
||||
|
||||
`sub2api-egress-proxy` 的持久化配置从 `config/platform-infra/sub2api.yaml` 渲染,运行面变更必须通过受控 CLI 落地。不要把一次性 `kubectl patch` 留作最终状态。
|
||||
|
||||
```bash
|
||||
|
||||
@@ -89,7 +89,9 @@ description: UniDesk YAML-first 运维正规化技能。用户提到 ymal-first/
|
||||
- 不得生成不可见默认值,也不得把合并后的大对象写成新的 source of truth。
|
||||
- 新 VPS 或裸节点 bootstrap 的出网必须优先走 YAML-first 声明的 0 依赖 host proxy client:通过 `trans` 分发零依赖客户端,先让 host route 能出网,再安装 k3s、containerd、Docker、包管理器依赖和 Git/gitrepo 同步;不得把 provider-gateway WebSocket egress 当作大二进制或依赖下载的数据面。
|
||||
- 多主机 k3s 集群:
|
||||
- 跨主机拓扑、WireGuard、agent token、registry mirror、ServiceLB 节点和调度 smoke 归独立集群 owning YAML。
|
||||
- 跨主机拓扑、WireGuard、agent token、worker host proxy 引用、集群内 artifact registry 映射、ServiceLB 节点和调度 smoke 归独立集群 owning YAML。
|
||||
- worker host proxy 归 `config/platform-infra/host-proxy.yaml`;集群 apply 必须先确认 proxy client 连接声明的 `vpn-server` 并通过外网 probe,再启动 agent。
|
||||
- containerd 通过 host proxy 直接访问公共 registry,禁止为公共依赖新增 registry mirror;集群内 artifact registry 映射不属于公共镜像代理。
|
||||
- 既有 control-plane 的 k3s renderer 通过 `clusterConfigRef` 消费集群网络参数,不得由集群 CLI 再写第二个 systemd drop-in。
|
||||
- 无固定公网地址 worker 使用宿主内网 SSH 作为部署控制通道并主动连接 WireGuard endpoint,不使用 FRP 代替三层集群网络。
|
||||
- control-plane drop-in 只走 `hwlab nodes control-plane infra k3s-config` 窄域:
|
||||
|
||||
@@ -82,6 +82,19 @@ targets:
|
||||
noProxyRefs:
|
||||
common: noProxy.common
|
||||
extras: noProxy.nodeExtras.NC01
|
||||
D601-VM:
|
||||
extends: templates.targets.hostProxyClient
|
||||
route: D601-VM
|
||||
sourceRef: sources.host-only-vpn-server-shadowsocks
|
||||
files:
|
||||
k3sSystemdDropIn: /etc/systemd/system/k3s-agent.service.d/10-unidesk-proxy.conf
|
||||
env:
|
||||
noProxyRefs:
|
||||
common: noProxy.common
|
||||
extras: noProxy.nodeExtras.D601-VM
|
||||
apply:
|
||||
restartDocker: false
|
||||
restartK3s: false
|
||||
G14-WSL:
|
||||
extends: templates.targets.hostProxyClient
|
||||
route: G14-WSL
|
||||
@@ -193,6 +206,8 @@ noProxy:
|
||||
- 152-53-229-148.nip.io
|
||||
NC01:
|
||||
- 152.53.229.148
|
||||
D601-VM:
|
||||
- 152.53.229.148
|
||||
G14-WSL:
|
||||
- 152.53.229.148
|
||||
JD01:
|
||||
|
||||
@@ -19,6 +19,7 @@ targets:
|
||||
D601-VM:
|
||||
id: D601-VM
|
||||
bootstrapConfigRef: config/platform-infra/hyperv-vms.yaml#targets.D601-VM
|
||||
hostProxyConfigRef: config/platform-infra/host-proxy.yaml#targets.D601-VM
|
||||
nodeName: d601-vm
|
||||
labels:
|
||||
unidesk.ai/node-id: D601-VM
|
||||
@@ -54,5 +55,5 @@ targets:
|
||||
smoke:
|
||||
namespace: platform-infra-smoke
|
||||
jobName: d601-runtime-plane-smoke
|
||||
image: docker.m.daocloud.io/library/busybox:1.36.1
|
||||
image: docker.io/library/busybox:1.36.1
|
||||
timeoutSeconds: 180
|
||||
|
||||
@@ -13,13 +13,15 @@
|
||||
## 多主机 k3s 集群
|
||||
|
||||
- 权威配置:
|
||||
- 跨主机拓扑、WireGuard、agent token `sourceRef`、registry mirror、ServiceLB 节点和 smoke 归 `config/platform-infra/k3s-clusters.yaml`。
|
||||
- 跨主机拓扑、WireGuard、agent token `sourceRef`、worker host proxy 引用、集群内 artifact registry 映射、ServiceLB 节点和 smoke 归 `config/platform-infra/k3s-clusters.yaml`。
|
||||
- worker host proxy 的服务端来源、客户端工件、Secret 引用、代理环境和 systemd drop-in 归 `config/platform-infra/host-proxy.yaml`。
|
||||
- Hyper-V VM 资源和宿主内网 SSH 归 `config/platform-infra/hyperv-vms.yaml`。
|
||||
- 既有 control-plane 的 k3s systemd drop-in 继续归 `config/hwlab-node-control-plane.yaml`,并通过 `clusterConfigRef` 消费集群网络参数。
|
||||
- 受控入口:
|
||||
- `platform-infra k3s-cluster secret-init --target <id> --confirm` 只在 Secret source 不存在时生成 WireGuard key pair 和 k3s agent token,存在时拒绝覆盖。
|
||||
- `platform-infra k3s-cluster plan|apply|status|smoke --target <id>` 是部署和验收入口。
|
||||
- 无固定公网地址的 worker 由宿主内网 SSH 控制,主动连接 control-plane WireGuard endpoint;worker provider route 只作为部署后补充验收入口。
|
||||
- 集群 apply 先经宿主内网 SSH 分发零依赖 proxy client,确认它连接声明的 `vpn-server` 且外网 probe 成功,再启动或恢复 `k3s-agent`。
|
||||
- control-plane 节点配置只组合 `hwlab nodes control-plane infra k3s-config plan|status|apply`:
|
||||
- 该窄域只比较并安装 owning YAML 渲染的 k3s systemd drop-in;
|
||||
- 只在 drop-in 或节点 pod 容量需要收敛时重启 k3s;
|
||||
@@ -30,17 +32,18 @@
|
||||
- worker 只运行 `k3s-agent`;加入前必须卸载旧独立 server,禁止保留第二 datastore/control-plane。
|
||||
- Flannel 使用 WireGuard 接口承载 VXLAN,不能只用 FRP 转发 Kubernetes API。
|
||||
- ServiceLB 只在 owning YAML 指定的公网节点启用,防止无公网地址 worker 接管 hostPort。
|
||||
- `127.0.0.1:5000` 镜像引用必须通过 worker 的 `registries.yaml` 映射到集群可达 registry endpoint。
|
||||
- worker 的 containerd 外网访问继承 host proxy systemd 环境,直接访问上游 registry;禁止为公共依赖新增 registry mirror。
|
||||
- `127.0.0.1:5000` 只表示 UniDesk 集群内 artifact registry,worker 的 `registries.yaml` 可把该本地引用映射到集群 endpoint,不得把它扩展为公共镜像代理。
|
||||
- 幂等重复执行不得强写:
|
||||
- WireGuard、systemd drop-in、agent unit、token 文件和节点标签均先比较声明态与运行态;
|
||||
- 内容一致时保持 `mutation=false`,不得无条件重启、重装或覆盖;
|
||||
- WireGuard 只在配置变化或 unit 未运行时重启/启动,避免短暂断网触发 control-plane 级联重启;
|
||||
- worker agent 以 owning YAML、版本、参数、registry mirror 和 token 的脱敏摘要判断是否需要重装;
|
||||
- worker agent 以 owning YAML、版本、参数、集群内 registry 映射和 token 的脱敏摘要判断是否需要重装;
|
||||
- worker 安装只做短启动,终态由 control-plane 的 Node Ready 轮询判定,不得让宿主 SSH 会话等待 systemd notify;
|
||||
- k3s API 暂时不可读属于 unknown,不得当作容量漂移触发 restart;
|
||||
- 集群 CLI 只组合各自 owning domain,不借用全量基础设施入口扩大写入范围。
|
||||
- 验收:
|
||||
- `status` 必须同时证明 WireGuard handshake、server/agent systemd、两个 Ready node、worker InternalIP 和旧 server 退役。
|
||||
- `status` 必须同时证明 WireGuard handshake、worker host proxy、server/agent systemd、两个 Ready node、worker InternalIP 和旧 server 退役。
|
||||
- `smoke --confirm` 必须把 Job 定向调度到 worker,并验证集群 DNS 和 Kubernetes ClusterIP;成功后删除 Job。
|
||||
- Secret 输出只显示 sourceRef、presence、key 名和 fingerprint,不显示值。
|
||||
## Hyper-V Ubuntu Provider VM
|
||||
|
||||
@@ -82,5 +82,6 @@ Provider TCP 数据池恢复必须满足:
|
||||
- Hyper-V 节点只能安装为 agent,不得保留独立 server。
|
||||
- 无固定公网地址的 worker 通过主动拨号的原生 WireGuard 三层隧道接入控制面。
|
||||
- WireGuard 私钥和 k3s agent token 只能来自 YAML `sourceRef`,不得从运行面反解或打印。
|
||||
- k3s 节点地址、Flannel 接口、API TLS SAN、registry mirror 和 ServiceLB 节点选择必须由 owning YAML 渲染。
|
||||
- k3s 节点地址、Flannel 接口、API TLS SAN、集群内 artifact registry 映射和 ServiceLB 节点选择必须由 owning YAML 渲染。
|
||||
- worker 必须先受控部署 host proxy client 并连接 YAML 引用的 `vpn-server`,再启动 agent;containerd 直接通过该 proxy 访问公共 registry,不得依赖公共镜像站或新增 registry mirror。
|
||||
- 宿主内网 SSH 可以作为 worker 的稳定部署控制通道,provider route 只作为部署后验收入口。
|
||||
|
||||
@@ -134,6 +134,20 @@ interface HostProxySecretMaterial {
|
||||
valuesPrinted: false;
|
||||
}
|
||||
|
||||
export interface HostProxyGuestBundle {
|
||||
targetId: string;
|
||||
configRef: string;
|
||||
localArchivePath: string;
|
||||
remoteArchivePath: string;
|
||||
archiveSha256: string;
|
||||
prepareScript: string;
|
||||
applyScript: string;
|
||||
statusScript: string;
|
||||
sourceFingerprint: string;
|
||||
materialFingerprint: string;
|
||||
valuesPrinted: false;
|
||||
}
|
||||
|
||||
type HostProxyAction = "plan" | "apply" | "status";
|
||||
|
||||
interface HostProxyOptions {
|
||||
@@ -160,6 +174,40 @@ export async function runPlatformInfraHostProxyCommand(_config: UniDeskConfig, a
|
||||
return renderApply(apply(config.server, target));
|
||||
}
|
||||
|
||||
export function prepareHostProxyGuestBundle(targetId: string): HostProxyGuestBundle {
|
||||
const config = readHostProxyConfig();
|
||||
const target = resolveTarget(config, targetId);
|
||||
if (target.source.client.mode !== "trans-static-binary") {
|
||||
throw new Error(`${configPath}.targets.${target.id} must use trans-static-binary`);
|
||||
}
|
||||
const material = proxySecretMaterial(config.server, target.source);
|
||||
const artifact = prepareClientArtifact(target.source.client);
|
||||
if (artifact.ok !== true) throw new Error(`host proxy artifact preparation failed for ${target.id}`);
|
||||
return {
|
||||
targetId,
|
||||
configRef: `${configPath}#targets.${targetId}`,
|
||||
localArchivePath: rootPath(target.source.client.archiveCachePath),
|
||||
remoteArchivePath: target.source.client.archiveInstallPath,
|
||||
archiveSha256: target.source.client.archiveSha256,
|
||||
prepareScript: remotePrepareScript(target.source.client),
|
||||
applyScript: `${remoteApplyScript(target, material.clientConfigJson)}
|
||||
if [ "$changed" = true ] && [ -s /etc/systemd/system/k3s-agent.service ]; then
|
||||
systemctl daemon-reload
|
||||
systemctl restart --no-block k3s-agent
|
||||
fi
|
||||
`,
|
||||
statusScript: remoteStatusScript(target),
|
||||
sourceFingerprint: target.source.fingerprint,
|
||||
materialFingerprint: material.fingerprint,
|
||||
valuesPrinted: false,
|
||||
};
|
||||
}
|
||||
|
||||
export function hostProxyGuestStatusScript(targetId: string): string {
|
||||
const config = readHostProxyConfig();
|
||||
return remoteStatusScript(resolveTarget(config, targetId));
|
||||
}
|
||||
|
||||
function parseOptions(args: string[]): HostProxyOptions {
|
||||
const actionRaw = args[0] ?? "plan";
|
||||
if (actionRaw !== "plan" && actionRaw !== "apply" && actionRaw !== "status") {
|
||||
@@ -758,11 +806,15 @@ function remoteApplyScript(target: HostProxyTarget, clientConfigJson: string): s
|
||||
const k3sDropIn = renderSystemdDropIn(target.files.envFile);
|
||||
const unit = renderClientUnit(client);
|
||||
const daemonReload = target.serviceManager === "systemd" && target.apply.reloadSystemd
|
||||
? "systemctl daemon-reload >/dev/null 2>&1 || true"
|
||||
? '[ "$changed" = true ] && systemctl daemon-reload >/dev/null 2>&1 || true'
|
||||
: ": # systemd reload disabled by YAML service manager";
|
||||
const startClient = target.serviceManager === "systemd"
|
||||
? `systemctl enable --now ${shQuote(client.serviceName)} >/tmp/unidesk-host-proxy-systemctl.out 2>/tmp/unidesk-host-proxy-systemctl.err || true
|
||||
systemctl restart ${shQuote(client.serviceName)} >/tmp/unidesk-host-proxy-restart.out 2>/tmp/unidesk-host-proxy-restart.err || true`
|
||||
? `systemctl enable ${shQuote(client.serviceName)} >/tmp/unidesk-host-proxy-systemctl.out 2>/tmp/unidesk-host-proxy-systemctl.err || true
|
||||
if [ "$changed" = true ]; then
|
||||
systemctl restart ${shQuote(client.serviceName)} >/tmp/unidesk-host-proxy-restart.out 2>/tmp/unidesk-host-proxy-restart.err || true
|
||||
elif ! systemctl is-active --quiet ${shQuote(client.serviceName)}; then
|
||||
systemctl start ${shQuote(client.serviceName)} >/tmp/unidesk-host-proxy-start.out 2>/tmp/unidesk-host-proxy-start.err || true
|
||||
fi`
|
||||
: processManagerStartShell(client);
|
||||
const restartDocker = target.apply.restartDocker
|
||||
? "nohup sh -c 'sleep 1; systemctl restart docker >/tmp/unidesk-host-proxy-docker-restart.out 2>/tmp/unidesk-host-proxy-docker-restart.err || true' >/dev/null 2>&1 &"
|
||||
@@ -772,6 +824,7 @@ systemctl restart ${shQuote(client.serviceName)} >/tmp/unidesk-host-proxy-restar
|
||||
: ": # restartK3s disabled by YAML";
|
||||
return `
|
||||
set -eu
|
||||
changed=false
|
||||
write_b64() {
|
||||
path="$1"
|
||||
mode="$2"
|
||||
@@ -780,7 +833,12 @@ write_b64() {
|
||||
tmp="$path.tmp.$$"
|
||||
printf '%s' "$data" | base64 -d >"$tmp"
|
||||
chmod "$mode" "$tmp"
|
||||
mv "$tmp" "$path"
|
||||
if cmp -s "$tmp" "$path" 2>/dev/null; then
|
||||
rm -f "$tmp"
|
||||
else
|
||||
mv "$tmp" "$path"
|
||||
changed=true
|
||||
fi
|
||||
}
|
||||
write_b64 ${shQuote(client.configPath)} 0600 ${shQuote(b64(clientConfigJson))}
|
||||
write_b64 ${shQuote(client.unitPath)} 0644 ${shQuote(b64(unit))}
|
||||
@@ -794,17 +852,21 @@ if [ "$archive_sha" != ${shQuote(client.archiveSha256)} ]; then
|
||||
printf 'archive sha256 mismatch\\n' >&2
|
||||
exit 23
|
||||
fi
|
||||
tmp_bin="${client.installPath}.tmp.$$"
|
||||
tar -xOf ${shQuote(client.archiveInstallPath)} ${shQuote(client.binaryMember)} >"$tmp_bin"
|
||||
chmod 0755 "$tmp_bin"
|
||||
binary_sha="$(sha256sum "$tmp_bin" | awk '{print $1}')"
|
||||
if [ "$binary_sha" != ${shQuote(client.binarySha256)} ]; then
|
||||
rm -f "$tmp_bin"
|
||||
printf 'binary sha256 mismatch\\n' >&2
|
||||
exit 24
|
||||
installed_binary_sha="$(sha256sum ${shQuote(client.installPath)} 2>/dev/null | awk '{print $1}' || true)"
|
||||
if [ "$installed_binary_sha" != ${shQuote(client.binarySha256)} ]; then
|
||||
tmp_bin="${client.installPath}.tmp.$$"
|
||||
tar -xOf ${shQuote(client.archiveInstallPath)} ${shQuote(client.binaryMember)} >"$tmp_bin"
|
||||
chmod 0755 "$tmp_bin"
|
||||
binary_sha="$(sha256sum "$tmp_bin" | awk '{print $1}')"
|
||||
if [ "$binary_sha" != ${shQuote(client.binarySha256)} ]; then
|
||||
rm -f "$tmp_bin"
|
||||
printf 'binary sha256 mismatch\\n' >&2
|
||||
exit 24
|
||||
fi
|
||||
mv -f "$tmp_bin" ${shQuote(client.installPath)}
|
||||
chmod 0755 ${shQuote(client.installPath)}
|
||||
changed=true
|
||||
fi
|
||||
mv -f "$tmp_bin" ${shQuote(client.installPath)}
|
||||
chmod 0755 ${shQuote(client.installPath)}
|
||||
${daemonReload}
|
||||
${startClient}
|
||||
sleep 2
|
||||
|
||||
@@ -18,6 +18,8 @@ export interface K3sClusterTarget {
|
||||
worker: {
|
||||
id: string;
|
||||
bootstrapConfigRef: string;
|
||||
hostProxyConfigRef: string;
|
||||
hostProxyTargetId: string;
|
||||
nodeName: string;
|
||||
labels: Record<string, string>;
|
||||
windowsRoute: string;
|
||||
@@ -97,6 +99,8 @@ export function readK3sClusterConfig(targetId?: string): { defaultTargetId: stri
|
||||
worker: {
|
||||
id: text(worker, "id", `targets.${id}.workers.${workerId}`),
|
||||
bootstrapConfigRef: text(worker, "bootstrapConfigRef", `targets.${id}.workers.${workerId}`),
|
||||
hostProxyConfigRef: text(worker, "hostProxyConfigRef", `targets.${id}.workers.${workerId}`),
|
||||
hostProxyTargetId: hostProxyTargetId(text(worker, "hostProxyConfigRef", `targets.${id}.workers.${workerId}`)),
|
||||
nodeName: text(worker, "nodeName", `targets.${id}.workers.${workerId}`),
|
||||
labels,
|
||||
windowsRoute: text(host, "route", "worker bootstrap host"),
|
||||
@@ -198,3 +202,9 @@ function stringMap(value: Record<string, unknown>, label: string): Record<string
|
||||
return [key, item];
|
||||
}));
|
||||
}
|
||||
|
||||
function hostProxyTargetId(ref: string): string {
|
||||
const match = ref.match(/^config\/platform-infra\/host-proxy\.yaml#targets\.([A-Za-z0-9._-]+)$/u);
|
||||
if (match === null) throw new Error(`worker hostProxyConfigRef must point at config/platform-infra/host-proxy.yaml#targets.<id>`);
|
||||
return match[1] ?? "";
|
||||
}
|
||||
|
||||
@@ -6,6 +6,11 @@ import { rootPath } from "./config";
|
||||
import { runCommand } from "./command";
|
||||
import { parseEnvFile } from "./secrets";
|
||||
import { runHwlabNodeControlPlaneInfra } from "./hwlab-node-control-plane";
|
||||
import {
|
||||
hostProxyGuestStatusScript,
|
||||
prepareHostProxyGuestBundle,
|
||||
type HostProxyGuestBundle,
|
||||
} from "./platform-infra-host-proxy";
|
||||
import { address, K3S_CLUSTER_CONFIG_LABEL, readK3sClusterConfig, type K3sClusterTarget } from "./platform-infra-k3s-cluster-config";
|
||||
|
||||
type Action = "plan" | "secret-init" | "apply" | "status" | "smoke";
|
||||
@@ -53,7 +58,13 @@ function plan(target: K3sClusterTarget): Record<string, unknown> {
|
||||
topology: {
|
||||
clusterName: target.clusterName,
|
||||
controlPlane: { id: target.controlPlane.id, nodeName: target.controlPlane.nodeName, route: target.controlPlane.route, kubeRoute: target.controlPlane.kubeRoute },
|
||||
worker: { id: target.worker.id, nodeName: target.worker.nodeName, bootstrapConfigRef: target.worker.bootstrapConfigRef, controlRoute: target.worker.windowsRoute },
|
||||
worker: {
|
||||
id: target.worker.id,
|
||||
nodeName: target.worker.nodeName,
|
||||
bootstrapConfigRef: target.worker.bootstrapConfigRef,
|
||||
hostProxyConfigRef: target.worker.hostProxyConfigRef,
|
||||
controlRoute: target.worker.windowsRoute,
|
||||
},
|
||||
},
|
||||
wireguard: { ...target.wireguard, endpoint: target.wireguard.endpoint },
|
||||
k3s: {
|
||||
@@ -101,6 +112,10 @@ function apply(target: K3sClusterTarget): Record<string, unknown> {
|
||||
]) as Record<string, unknown>;
|
||||
if (controlPlane.ok !== true) throw new Error(`control-plane-k3s-apply failed: ${bounded(JSON.stringify(compactDomain(controlPlane)))}`);
|
||||
|
||||
const hostProxy = prepareHostProxyGuestBundle(target.worker.hostProxyTargetId);
|
||||
const workerHostProxy = runWorkerHostProxy(target, hostProxy, 240_000);
|
||||
requireSuccess(workerHostProxy, "worker-host-proxy-apply");
|
||||
|
||||
const worker = runWorker(target, workerApplyScript(target, secret), true, 300_000);
|
||||
requireSuccess(worker, "worker-agent-apply");
|
||||
|
||||
@@ -120,6 +135,13 @@ function apply(target: K3sClusterTarget): Record<string, unknown> {
|
||||
stages: {
|
||||
controlPlaneWireguard: compact(controlWireguard),
|
||||
controlPlaneK3s: compactDomain(controlPlane),
|
||||
workerHostProxy: {
|
||||
...compact(workerHostProxy),
|
||||
configRef: hostProxy.configRef,
|
||||
sourceFingerprint: hostProxy.sourceFingerprint,
|
||||
materialFingerprint: hostProxy.materialFingerprint,
|
||||
valuesPrinted: false,
|
||||
},
|
||||
workerAgent: compact(worker),
|
||||
serviceLbPlacement: compact(labelResult),
|
||||
nodeReadiness: compact(ready),
|
||||
@@ -134,18 +156,26 @@ function apply(target: K3sClusterTarget): Record<string, unknown> {
|
||||
|
||||
function status(target: K3sClusterTarget): Record<string, unknown> {
|
||||
const control = runControlPlane(target, statusScript(target), 60_000);
|
||||
const workerProxy = runWorker(target, hostProxyGuestStatusScript(target.worker.hostProxyTargetId), false, 60_000);
|
||||
const worker = runWorker(target, workerStatusScript(target), false, 60_000);
|
||||
const controlData = parseFirstJson(control.stdout);
|
||||
const workerProxyData = parseFirstJson(workerProxy.stdout);
|
||||
const workerData = parseFirstJson(worker.stdout);
|
||||
const ok = control.exitCode === 0 && worker.exitCode === 0 && controlData.ok === true && workerData.ok === true;
|
||||
const ok = control.exitCode === 0
|
||||
&& workerProxy.exitCode === 0
|
||||
&& worker.exitCode === 0
|
||||
&& controlData.ok === true
|
||||
&& workerProxyData.ok === true
|
||||
&& workerData.ok === true;
|
||||
return {
|
||||
ok,
|
||||
mutation: false,
|
||||
targetId: target.id,
|
||||
controlPlane: controlData,
|
||||
workerHostProxy: workerProxyData,
|
||||
worker: workerData,
|
||||
secret: secretSummary(target),
|
||||
result: { controlPlane: compact(control), worker: compact(worker) },
|
||||
result: { controlPlane: compact(control), workerHostProxy: compact(workerProxy), worker: compact(worker) },
|
||||
next: ok ? { smoke: `bun scripts/cli.ts platform-infra k3s-cluster smoke --target ${target.id} --confirm` } : { apply: `bun scripts/cli.ts platform-infra k3s-cluster apply --target ${target.id} --confirm` },
|
||||
};
|
||||
}
|
||||
@@ -411,6 +441,49 @@ function runWorker(target: K3sClusterTarget, script: string, copyArtifacts: bool
|
||||
return runCommand(["trans", target.worker.windowsRoute, "ps"], rootPath(), { input: lines.join("\n") + "\n", timeoutMs });
|
||||
}
|
||||
|
||||
function runWorkerHostProxy(target: K3sClusterTarget, bundle: HostProxyGuestBundle, timeoutMs: number) {
|
||||
const windowsArchive = `${target.worker.stateDir}\\host-proxy-${bundle.targetId}.tar.gz`;
|
||||
const upload = runCommand(
|
||||
["trans", target.worker.windowsRoute, "upload", bundle.localArchivePath, windowsArchive],
|
||||
rootPath(),
|
||||
{ timeoutMs },
|
||||
);
|
||||
if (upload.exitCode !== 0) return upload;
|
||||
|
||||
const guestArchive = `/tmp/unidesk-host-proxy-${bundle.targetId}.tar.gz`;
|
||||
const remoteScript = `/tmp/unidesk-host-proxy-${bundle.targetId}.sh`;
|
||||
const script = `${bundle.prepareScript}
|
||||
remote_archive_sha="$(sha256sum ${shQuote(bundle.remoteArchivePath)} 2>/dev/null | awk '{print $1}' || true)"
|
||||
if [ "$remote_archive_sha" != ${shQuote(bundle.archiveSha256)} ]; then
|
||||
install -m 0644 ${shQuote(guestArchive)} ${shQuote(bundle.remoteArchivePath)}
|
||||
fi
|
||||
${bundle.applyScript}
|
||||
`;
|
||||
const encoded = Buffer.from(script, "utf8").toString("base64");
|
||||
const keyPath = `${target.worker.stateDir}\\id_ed25519`;
|
||||
const lines = [
|
||||
"$ErrorActionPreference='Stop'",
|
||||
`$key=${psQuote(keyPath)}`,
|
||||
`$guest=${psQuote(`${target.worker.guestUser}@${target.worker.guestAddress}`)}`,
|
||||
`$archive=${psQuote(windowsArchive)}`,
|
||||
`$temp=Join-Path $env:TEMP ${psQuote(`unidesk-host-proxy-${bundle.targetId}.sh`)}`,
|
||||
`[IO.File]::WriteAllBytes($temp,[Convert]::FromBase64String(${psQuote(encoded)}))`,
|
||||
`& scp.exe -i $key -o BatchMode=yes -o StrictHostKeyChecking=accept-new $archive "$($guest):${guestArchive}"`,
|
||||
"if($LASTEXITCODE -ne 0){throw 'worker-host-proxy-archive-scp-failed'}",
|
||||
`& scp.exe -i $key -o BatchMode=yes -o StrictHostKeyChecking=accept-new $temp "$($guest):${remoteScript}"`,
|
||||
"if($LASTEXITCODE -ne 0){throw 'worker-host-proxy-script-scp-failed'}",
|
||||
`& ssh.exe -i $key -o BatchMode=yes -o StrictHostKeyChecking=accept-new -o ServerAliveInterval=15 $guest ${psQuote(`sudo sh ${remoteScript}`)}`,
|
||||
"$rc=$LASTEXITCODE",
|
||||
"Remove-Item -Force -LiteralPath $temp -ErrorAction SilentlyContinue",
|
||||
`& ssh.exe -i $key -o BatchMode=yes -o StrictHostKeyChecking=accept-new $guest ${psQuote(`rm -f ${remoteScript} ${guestArchive}`)} | Out-Null`,
|
||||
"exit $rc",
|
||||
];
|
||||
return runCommand(["trans", target.worker.windowsRoute, "ps"], rootPath(), {
|
||||
input: lines.join("\n") + "\n",
|
||||
timeoutMs,
|
||||
});
|
||||
}
|
||||
|
||||
function readSecrets(target: K3sClusterTarget): SecretMaterial {
|
||||
if (!existsSync(target.secrets.sourceRef)) throw new Error(`${target.secrets.sourceRef} is missing; run platform-infra k3s-cluster secret-init --confirm`);
|
||||
const values = parseEnvFile(readFileSync(target.secrets.sourceRef, "utf8"));
|
||||
|
||||
Reference in New Issue
Block a user