feat(ci): eval matrix for opt-in nomarchy.* toggles
nix flake check --no-build only evaluates the four default configs, so a bug that only fires when a nomarchy.* toggle is flipped sails through — exactly how the vscode option rename and the impermanence systemd-stage-1 assertion both reached main undetected. Add bin/utils/nomarchy-eval-matrix: layers each opt-in scenario (16 toggles, both system and home halves) onto nixosConfigurations.default via extendModules and forces system.build.toplevel.drvPath (which forces assertion checks); also forces all 21 palettes through the system-side Plymouth base00->RGB math that the home-only allThemeVariants never reaches, plus both standalone homeConfigurations. Per-scenario results come from builtins.tryEval so one failure doesn't mask the rest; verified it goes red on a re-injected assertion failure and a bad option ref, green on the current tree. Wired into .forgejo/workflows/check.yml after flake check (jq via nix shell so it doesn't depend on the runner image). AGENT.md §5 now tells future agents to add a scenario when they add/rename an opt-in option. nomarchy-docs-scripts: filter systemd-service declarations (.services. nomarchy-) and denylist the eval-matrix repo tool so neither shows as a dangling "missing" reference; also retroactively clears the latent nomarchy-flathub-init drift from the gaming-flathub commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Bernardo Magri
@@ -7,6 +7,11 @@
|
||||
# 3. `docs/SCRIPTS.md` drifts from the repo state because somebody
|
||||
# added / removed / renamed a script and didn't run the generator
|
||||
# (the pre-commit hook handles this, but only when enabled per-clone).
|
||||
# 4. An opt-in nomarchy.* toggle stops evaluating. `nix flake check`
|
||||
# only evaluates the four default configs, so a bug that only fires
|
||||
# when a toggle is flipped sails through (e.g. the impermanence
|
||||
# systemd-stage-1 assertion). nomarchy-eval-matrix layers each toggle
|
||||
# onto the default config and forces system.build.toplevel.
|
||||
#
|
||||
# Doesn't build ISOs — that needs a binary cache. Add a separate job
|
||||
# once Cachix/Attic is in place.
|
||||
@@ -36,6 +41,12 @@ jobs:
|
||||
- name: nix flake check --no-build
|
||||
run: nix flake check --no-build
|
||||
|
||||
- name: Evaluate opt-in toggle / palette / home matrix
|
||||
# flake check never flips a nomarchy.* toggle; this does. jq is
|
||||
# provided via nix shell so the step doesn't depend on the runner
|
||||
# image preinstalling it.
|
||||
run: nix shell nixpkgs#jq --command ./bin/utils/nomarchy-eval-matrix
|
||||
|
||||
- name: Lint nomarchy-* scripts (bash -n + shellcheck)
|
||||
run: |
|
||||
# Mirror what .githooks/pre-commit runs locally, but across the
|
||||
|
||||
@@ -98,10 +98,12 @@ all_refs=$(grep -rhE 'nomarchy-[a-z0-9]([a-z0-9-]*[a-z0-9])?' \
|
||||
-e '\./result/bin/run-nomarchy-' \
|
||||
-e 'mktemp[[:space:]]+[^|]*-t[[:space:]]+nomarchy-' \
|
||||
-e '(TIMER_NAME|NOPASSWD_FILE|UNIT_NAME)=.*nomarchy-' \
|
||||
-e '\.services\.nomarchy-' \
|
||||
-e 'docker[[:space:]]+[^|]*nomarchy-' \
|
||||
| grep -oE 'nomarchy-[a-z0-9]([a-z0-9-]*[a-z0-9])?' \
|
||||
| grep -vE '^(nomarchy-launch|nomarchy-brightness|nomarchy-cmd|nomarchy-pkg|nomarchy-restart|nomarchy-toggle|nomarchy-theme|nomarchy-webapp-handler|nomarchy-font-selector|nomarchy-theme-selector|nomarchy-wallpaper-selector|nomarchy-setup|nomarchy-refresh|nomarchy-scripts|nomarchy-system-scripts|nomarchy-theme-engine-scripts)$' \
|
||||
| grep -vE '^(nomarchy-plymouth|nomarchy-sddm-theme|nomarchy-live|nomarchy-rev|nomarchy-windows)$' \
|
||||
| grep -vE '^(nomarchy-eval-matrix)$' \
|
||||
| sort -u)
|
||||
# The second denylist covers identifiers whose ambiguity survives the line
|
||||
# filter: `nomarchy-plymouth` / `nomarchy-sddm-theme` are Nix derivation
|
||||
@@ -109,6 +111,9 @@ all_refs=$(grep -rhE 'nomarchy-[a-z0-9]([a-z0-9-]*[a-z0-9])?' \
|
||||
# ISO label that shows up in comments, `nomarchy-rev` is `/etc/nomarchy-rev`
|
||||
# (written by the ISO), and `nomarchy-windows` is a docker container name
|
||||
# in compose heredocs.
|
||||
# The third denylist: `nomarchy-eval-matrix` is a bin/utils repo tool (like
|
||||
# this generator) that names itself in its own header comment — not a
|
||||
# user-facing script that ships in nomarchy-system-scripts.
|
||||
|
||||
# --- Render: header --------------------------------------------------------
|
||||
|
||||
|
||||
119
bin/utils/nomarchy-eval-matrix
Executable file
119
bin/utils/nomarchy-eval-matrix
Executable file
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env bash
|
||||
# nomarchy-eval-matrix
|
||||
#
|
||||
# Evaluates the opt-in nomarchy.* surface that `nix flake check` never
|
||||
# touches. flake check only evaluates the four default configs, so a bug
|
||||
# that only fires when a toggle is flipped — a renamed option, a failed
|
||||
# assertion, a stale reference — sails straight through. Two such bugs
|
||||
# shipped to main before this existed (the vscode option rename and the
|
||||
# impermanence systemd-stage-1 assertion); both would have been caught
|
||||
# here.
|
||||
#
|
||||
# Each toggle scenario is layered onto nixosConfigurations.default via
|
||||
# extendModules and its system.build.toplevel.drvPath is forced (which
|
||||
# also forces the assertion checks). Every palette is forced through the
|
||||
# system-side Plymouth colour math (which allThemeVariants, being
|
||||
# home-only, doesn't exercise). Both standalone homeConfigurations are
|
||||
# forced too.
|
||||
#
|
||||
# Per-scenario results come from builtins.tryEval so one failure doesn't
|
||||
# mask the rest — the whole matrix runs, then the script exits non-zero
|
||||
# if anything failed.
|
||||
#
|
||||
# nomarchy-eval-matrix # run the full matrix, table + exit code
|
||||
#
|
||||
# Add a scenario by editing the `toggleScenarios` attrset in the embedded
|
||||
# Nix expression below.
|
||||
|
||||
set -u
|
||||
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
NIX=(nix --extra-experimental-features 'nix-command flakes')
|
||||
|
||||
read -r -d '' EXPR <<'NIXEOF' || true
|
||||
let
|
||||
f = builtins.getFlake (toString ./.);
|
||||
lib = f.inputs.nixpkgs.lib;
|
||||
themeNames = (import ./lib { inherit lib; }).themeNames;
|
||||
default = f.nixosConfigurations.default;
|
||||
|
||||
# Opt-in toggles layered onto the default config. Home-side options go
|
||||
# under home-manager.users.nomarchy.nomarchy.*; system-side directly.
|
||||
toggleScenarios = {
|
||||
"impermanence-single" = { nomarchy.system.impermanence.enable = true; };
|
||||
"impermanence-multi" = { nomarchy.system.impermanence = { enable = true; mainLuksName = "crypted_main"; }; };
|
||||
"laptop-preset" = { nomarchy.system = { formFactor = "laptop"; laptop.enable = true; }; };
|
||||
"desktop-preset" = { nomarchy.system = { formFactor = "desktop"; desktop.enable = true; }; };
|
||||
"accessibility-system" = { nomarchy.system.accessibility.enable = true; };
|
||||
"gaming-system" = { nomarchy.system.gaming.enable = true; };
|
||||
"hybrid-gpu" = { nomarchy.system.features.hybridGPU = true; };
|
||||
"hibernation" = { nomarchy.system.hibernation.enable = true; };
|
||||
"docker" = { nomarchy.system.virtualization.docker.enable = true; };
|
||||
"fwupd" = { nomarchy.hardware.fwupd = true; };
|
||||
"accessibility-home" = { home-manager.users.nomarchy.nomarchy.accessibility.enable = true; };
|
||||
"gaming-home" = { home-manager.users.nomarchy.nomarchy.gaming.enable = true; };
|
||||
"overrides" = { home-manager.users.nomarchy.nomarchy.overrides.enable = true; };
|
||||
"panel-bottom" = { home-manager.users.nomarchy.nomarchy.panelPosition = "bottom"; };
|
||||
"keymap-variant" = { home-manager.users.nomarchy.nomarchy.keymap = { layout = "de"; variant = "nodeadkeys"; }; };
|
||||
"toggles-off" = { home-manager.users.nomarchy.nomarchy.toggles = { waybar = false; idle = false; nightlight = false; }; };
|
||||
};
|
||||
|
||||
forced = x: (builtins.tryEval (builtins.seq x true)).success;
|
||||
|
||||
tryToplevel = mod:
|
||||
forced (default.extendModules { modules = [ mod ]; }).config.system.build.toplevel.drvPath;
|
||||
|
||||
# Palettes only need the system-side Plymouth path forced — that's the
|
||||
# per-palette base00 → RGB float math in themes/engine/plymouth.nix that
|
||||
# the home-only allThemeVariants never reaches.
|
||||
tryPalette = name:
|
||||
forced (default.extendModules { modules = [ { nomarchy.system.theme = name; } ]; }).config.boot.plymouth.themePackages;
|
||||
|
||||
tryHome = name: forced f.homeConfigurations.${name}.activationPackage.drvPath;
|
||||
in {
|
||||
toggles = builtins.mapAttrs (_: tryToplevel) toggleScenarios;
|
||||
palettes = builtins.listToAttrs (map (n: { name = n; value = tryPalette n; }) themeNames);
|
||||
home = builtins.listToAttrs (map (n: { name = n; value = tryHome n; }) [ "nomarchy" "nixos" ]);
|
||||
}
|
||||
NIXEOF
|
||||
|
||||
echo "Evaluating opt-in toggle + palette + home matrix (this takes a few minutes)..."
|
||||
json="$("${NIX[@]}" eval --impure --json --expr "$EXPR" 2>/tmp/eval-matrix.err)"
|
||||
status=$?
|
||||
if [[ $status -ne 0 || -z "$json" ]]; then
|
||||
echo "ERROR: the matrix evaluation aborted before producing results." >&2
|
||||
echo "This usually means an uncatchable error (abort/infinite recursion) in one" >&2
|
||||
echo "scenario, or a syntax error in the expression. Full output:" >&2
|
||||
cat /tmp/eval-matrix.err >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Render each section and tally failures.
|
||||
fails=0
|
||||
render() {
|
||||
local section="$1"
|
||||
echo
|
||||
echo "=== $section ==="
|
||||
while IFS=$'\t' read -r name ok; do
|
||||
if [[ "$ok" == "true" ]]; then
|
||||
printf ' ok %s\n' "$name"
|
||||
else
|
||||
printf ' FAIL %s\n' "$name"
|
||||
fails=$((fails + 1))
|
||||
fi
|
||||
done < <(echo "$json" | jq -r --arg s "$section" '.[$s] | to_entries[] | "\(.key)\t\(.value)"' | sort)
|
||||
}
|
||||
|
||||
render toggles
|
||||
render palettes
|
||||
render home
|
||||
|
||||
echo
|
||||
if [[ $fails -gt 0 ]]; then
|
||||
echo "$fails scenario(s) failed to evaluate. Re-run the failing one directly for the trace, e.g.:"
|
||||
echo " nix eval --impure --show-trace --expr 'let f = builtins.getFlake (toString ./.); in (f.nixosConfigurations.default.extendModules { modules = [ { <the toggle> } ]; }).config.system.build.toplevel.drvPath'"
|
||||
exit 1
|
||||
fi
|
||||
echo "All scenarios evaluated cleanly."
|
||||
@@ -143,6 +143,11 @@ Steps you should follow for any non-trivial change:
|
||||
nix --extra-experimental-features 'nix-command flakes' flake check --no-build
|
||||
bash -n installer/install.sh # if you touched it
|
||||
```
|
||||
`flake check` only evaluates the four *default* configs — it never flips a `nomarchy.*` toggle. If you touched anything that only fires when a toggle is enabled (impermanence, a preset, an opt-in feature, the per-palette system theme), also run the toggle matrix, which layers each opt-in scenario onto the default config and forces it:
|
||||
```bash
|
||||
./bin/utils/nomarchy-eval-matrix
|
||||
```
|
||||
When you **add or rename an opt-in `nomarchy.*` option**, add a matching scenario to the `toggleScenarios` attrset in that script — otherwise the new surface is unverified and the next eval-time bug in it ships silently (this is exactly how the impermanence assertion and the vscode option rename reached `main`). The matrix also runs in CI (`.forgejo/workflows/check.yml`).
|
||||
First clone? Enable the repo's pre-commit hook so `docs/SCRIPTS.md` regenerates whenever you add/modify/remove a `nomarchy-*` script:
|
||||
```bash
|
||||
git config core.hooksPath .githooks
|
||||
|
||||
@@ -147,6 +147,8 @@ Pillar is **done** when every component has a closed `wave/qa-<component>` PR an
|
||||
|
||||
(Move items here when they land — keep them brief, link the commit/PR.)
|
||||
|
||||
- _2026-05-29_ — **Pillar 7: opt-in toggle eval matrix in CI.** `nix flake check --no-build` only evaluates the four default `nixosConfigurations`/`homeConfigurations`, so any eval-time bug that *only* fires when a `nomarchy.*` toggle is flipped sailed straight through — exactly how the vscode option rename and the impermanence systemd-stage-1 assertion both reached `main` undetected. New `bin/utils/nomarchy-eval-matrix` layers each opt-in scenario (16 toggles: impermanence single/multi, laptop/desktop presets, accessibility, gaming, hybridGPU, hibernation, docker, fwupd, overrides, panel position, keymap variant, toggles-off — plus home + system halves) onto `nixosConfigurations.default` via `extendModules` and forces `system.build.toplevel.drvPath` (which forces the assertion checks); it also forces all 21 palettes through the system-side Plymouth `base00`→RGB-float math that the home-only `allThemeVariants` never reaches, and both standalone `homeConfigurations`. Per-scenario results come from `builtins.tryEval` so one failure doesn't mask the rest — verified the harness goes red on both a failed assertion (re-injected `boot.initrd.postDeviceCommands`) and a nonexistent-option ref, and green on the current tree (all 39 scenarios pass). Wired into `.forgejo/workflows/check.yml` as a step after `flake check`, with `jq` supplied via `nix shell` so it doesn't depend on the runner image. `docs/AGENT.md` §5.5 now instructs: when you add or rename an opt-in `nomarchy.*` option, add a matching scenario to the script's `toggleScenarios` attrset.
|
||||
|
||||
- _2026-05-29_ — **Impermanence rollback no longer fails to build.** `core/system/impermanence.nix` implemented the Erase-Your-Darlings root wipe via `boot.initrd.postDeviceCommands`, but `themes/engine/plymouth.nix:63` sets `boot.initrd.systemd.enable = true` distro-wide (Plymouth is imported unconditionally from `core/system/default.nix`), and systemd stage-1 initrd hard-rejects `postDeviceCommands` with a failed assertion. Net effect: **every** install that flipped `nomarchy.system.impermanence.enable = true` — including any user who picked impermanence at the installer prompt — produced a config that wouldn't evaluate, let alone boot. Converted the rollback into a `boot.initrd.systemd.services.nomarchy-rollback` oneshot unit ordered `after = systemd-cryptsetup@${mainLuksName}.service` and `before = sysroot.mount`, with `boot.initrd.systemd.initrdBin` pulling in `btrfs-progs` / `coreutils` / `util-linux` / `findutils` (the systemd initrd doesn't ship them by default, unlike the old scripted initrd). Script body (timestamped `@` → `old_roots` move, 30-day recursive subvolume GC, `root-blank` → `@` snapshot) is unchanged. Verified the assertion is gone and `nixosConfigurations.default` + `impermanence.enable` evaluates fully via `extendModules`. **Runtime-verification caveat:** the boot-time wipe semantics (the `[[ ]]`/function/IFS shell idioms and the cleanup loop under the systemd initrd shell, plus correct ordering vs the LUKS mapping) can only be confirmed by a real wipe-boot cycle on an impermanence install — fold into the Pillar 8 punch-list. `docs/TROUBLESHOOTING.md` persistence-block line reference updated (46-72 → 91-120).
|
||||
|
||||
- _2026-05-22_ — **VSCode theme extensions pinned for 10 marketplace palettes.** Before this fix, only the 6 palettes whose theme extensions ship in `pkgs.vscode-extensions` (catppuccin, catppuccin-latte, nord, tokyo-night, rose-pine, gruvbox) had working VSCode theming — every other palette had `workbench.colorTheme` set to a theme VSCode couldn't find, so it silently fell back to the built-in default. Including the default `summer-night` palette (sainnhe.everforest), which meant the default install had broken VSCode theming. Probed all 13 unique extensions against the VSCode marketplace extensionquery API; 10 exist (`sainnhe.everforest`, `shadesOfBuntu.flexoki-light`, `qufiwefefwoyn.kanagawa`, `oldjobobo.{lumon,miasma,retro-82}-theme`, `TahaYVR.matteblack`, `jovejonovski.ocean-green`, `monokai.theme-monokai-pro-vscode`, `Bjarne.white-theme`) and 3 don't (`Bjarne.{ethereal,hackerman,vantablack}-nomarchy` — unpublished custom Nomarchy themes; logged as a new Later row). Fetched the latest version + sha256 for each of the 10 via `nix store prefetch-file` against `https://${publisher}.gallery.vsassets.io/_apis/public/gallery/publisher/${publisher}/extension/${name}/${version}/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage`. Added a `marketplaceExtensions` list to `features/apps/vscode.nix` that wraps each entry in `pkgs.vscode-utils.extensionFromVscodeMarketplace { publisher; name; version; sha256; }` and concatenates with the existing nixpkgs-packaged list. Smoke-built `sainnhe.everforest` end-to-end to confirm the URL pattern + sha256 resolve. Documented the version-bump procedure in the module comment. `docs/OPTIONS.md` updated to drop the "still break" caveat for everything except the three Bjarne palettes.
|
||||
|
||||
Reference in New Issue
Block a user