fix(ci): memory-bounded eval — one output per nix process
Some checks failed
Check / eval (push) Failing after 7m59s
Some checks failed
Check / eval (push) Failing after 7m59s
check.yml has been permanently red since the runner container was
capped at 2 GB (uncapped, it OOM'd the whole 4 GB VPS). Root cause,
measured with GNU time (eval-cache off): `nix flake check --no-build`
walks every output in ONE evaluator process and peaks at ~6.0 GB RSS
on this repo — the checks.* suite alone is ~15 runNixOSTests, each a
full NixOS eval, all accumulating in one heap.
New tools/ci-eval.sh: identical coverage (all checks.* drvPaths, both
nixosConfigurations toplevels, the HM activationPackage — enumerated
dynamically, no drift), one fresh nix process per output so the heap is
freed between outputs. Cold per-output peaks: worst check 0.99 GB
(hardware-toggles), nomarchy toplevel 0.78 GB, HM 0.60 GB; the one
outlier is nomarchy-live at 2.69 GB (live-ISO eval), which fits the
2 GB cap only via the container's swap allowance (docker's default
--memory-swap is 2x memory). check.yml and bump.yml both call the
script now; bump.yml also gains max-jobs=1/cores=2 (it had no limits,
so its V1 build gate could spawn parallel builders in the capped
container). ROADMAP's stale "no CI today" corrected: bump.yml landed
8fded63 on schedule 2026-07-06.
Verified: V1+ — ci-eval.sh green locally end-to-end (3m22s, tree peak
2.75 GB = nomarchy-live's process); per-output peaks measured
individually. The decisive proof is the next Actions run on the VPS
itself — watch the nomarchy-live step; if it OOMs there, the runner
needs swap allowed: --memory=2g --memory-swap=6g.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -19,6 +19,28 @@ Template:
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-12 — CI OOM: memory-bounded eval for check.yml/bump.yml
|
||||
- **Task:** Bernardo: check.yml always red since he capped the runner
|
||||
container at 2 GB (uncapped it OOM'd the whole 4 GB VPS); bump.yml
|
||||
status unknown.
|
||||
- **Did:** Measured (GNU time, eval-cache off): one-process `nix flake
|
||||
check --no-build` peaks **6.0 GB** RSS — that's the whole story. New
|
||||
`tools/ci-eval.sh`: same coverage, ONE output per nix process (heap
|
||||
freed between outputs); both workflows now call it. bump.yml also got
|
||||
max-jobs=1/cores=2. Facts: bump.yml WORKS — landed 8fded63 on
|
||||
2026-07-06, on schedule; ROADMAP's "no CI today" was stale → fixed.
|
||||
- **Verified:** V1+ — ci-eval.sh ran green locally end-to-end (3:22);
|
||||
per-output cold peaks measured: worst check 0.99 GB
|
||||
(hardware-toggles), nomarchy toplevel 0.78 GB, HM 0.60 GB, **except
|
||||
nomarchy-live 2.69 GB** (live ISO eval) — fits only with the
|
||||
container's swap allowance (docker default --memory-swap = 2×memory);
|
||||
if Bernardo set swap off, that one output still OOMs → server-side
|
||||
one-liner (--memory=2g --memory-swap=6g) or a skip-hatch decision.
|
||||
- **Pending:** real proof is the next push's Actions run on the VPS —
|
||||
watch nomarchy-live's step; can't be verified from this machine.
|
||||
- **Next suggestion:** confirm green run on the remote, then hardware
|
||||
V3 batch.
|
||||
|
||||
## 2026-07-12 — parallel fingerprint-or-password (interactive, promoted)
|
||||
- **Task:** Bernardo promoted the PROPOSED "fingerprint or password in
|
||||
parallel" item live; wants it default-on for fingerprint-PAM users.
|
||||
|
||||
Reference in New Issue
Block a user