Files
Nomarchy/docs/TESTING.md
Bernardo Magri a50e9793ea feat(tools): commit the VM test harness + one-command installer regression
Promotes the throwaway harness that verified the installer into tools/:
- tools/vm/qmp.py        QMP keystroke injection / typing / quit
- tools/vm/vncshot.py    GL-safe screenshots via VNC readback (QMP
                         screendump shows "no surface" with virtio-vga-gl)
- tools/vm/gap-analysis.py  drv-graph diff that converged the offline
                         pin set; run it when offline installs build
                         from source
- tools/test-install.sh  the full offline regression: build ISO, boot
                         offline, unattended LUKS+swap install via a
                         config disk (typed long commands drop keys),
                         wait for poweroff, boot the installed disk,
                         screenshot the first boot for visual verdict

docs/TESTING.md §4 now points at the script.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 08:05:13 +01:00

118 lines
5.5 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Testing Nomarchy
How to verify changes actually work — from cheap eval checks to booting the
full desktop. Adapted from the previous Nomarchy iteration's agent docs.
**The honesty rule (for humans and AI agents alike):** for Waybar, Hyprland,
or any visual change, the only reliable check is booting the live ISO. If
you can't boot it (e.g. you're on macOS or have no KVM), **say so** rather
than claiming success. "All Nix files parse" is not "the bar renders."
## 1. Cheap checks (always do these first)
```sh
nix flake check --no-build # full module-system evaluation, no builds
nix-instantiate --parse <file> # syntax-only, works even on macOS
python3 -m py_compile pkgs/nomarchy-theme-sync/nomarchy-theme-sync.py
```
`nix flake check` needs a Linux machine (or a Linux builder) since all
outputs are `x86_64-linux`. It catches type errors, missing options, and
bad merges — most breakage stops here. It also evaluates the downstream
template through `lib.mkFlake` (including a real nixos-hardware profile),
so template/wrapper drift fails fast too.
## 2. Build and boot the live ISO
```sh
# Build + boot in QEMU, one command:
tools/test-live-iso.sh
# Or by hand:
nix build .#nixosConfigurations.nomarchy-live.config.system.build.isoImage
# → result/iso/*.iso — dd it to a USB stick for real hardware
```
The script prefers UEFI (OVMF) with a legacy-BIOS fallback, uses KVM when
`/dev/kvm` is readable, and boots with `virtio-vga-gl` + `gl=on`
**Hyprland and Ghostty need real OpenGL in the guest**; without it the
session may not start.
### What the live environment gives you
- Auto-login as `nomarchy` (no password, passwordless sudo), greetd →
Hyprland via `initial_session`; logging out lands on tuigreet.
- The flake is seeded writable at `~/.nomarchy` (from the read-only
`/etc/nomarchy` copy) — `$NOMARCHY_PATH` already points there.
- Locked flake inputs are pinned into the ISO store, so
`home-manager switch` works **without a network**.
## 3. Verification checklist
Work through these in order; each one exercises a different layer.
| # | Check | Verifies |
|---|---|---|
| 1 | Boots to Hyprland with Tokyo Night wallpaper visible | greetd autologin, awww, session start |
| 2 | Waybar shows at top, themed (blue accent on dark) | HM waybar module, palette baking |
| 3 | `SUPER+Return` opens Ghostty with Tokyo Night colors | terminal default, ANSI palette |
| 4 | `btop` in the terminal is themed | per-theme asset baking |
| 5 | `nomarchy-theme-sync list` prints 21 presets | package, baked themes dir |
| 6 | `nomarchy-theme-sync apply gruvbox` → state written, `home-manager switch` runs, desktop re-themes, wallpaper changes | the whole engine: state write, pure eval, HM rebuild, wallpaper hook |
| 7 | `SUPER+SHIFT+T` cycles wallpapers instantly (try `tokyo-night`: 4 of them) | the runtime wallpaper path |
| 8 | `nomarchy-theme-sync apply summer-night` → after the switch the bar has its own identity (light bar, different styling) | whole-swap waybar.css assets |
| 9 | Open a GTK app — dark theme matching the palette | Stylix layer |
| 10 | `home-manager generations` lists one generation per theme change; activating an older one rolls the theme back | atomicity / rollback story |
Items 6 and 8 are the big ones — they prove the all-Home-Manager theming
model end to end.
## 4. Testing the installer
One command runs the whole offline regression:
```sh
tools/test-install.sh
```
It builds the ISO, boots it in QEMU **with networking disabled**, runs an
unattended LUKS+swap install onto a blank disk via the installer's
`NOMARCHY_UNATTENDED=1` mode (config delivered on a small vfat disk —
typing long commands into a guest drops keystrokes), waits for the
installer's poweroff, then boots the installed disk, enters the
passphrase, and screenshots the first boot. The machine-checkable parts
(install completes, disk boots) fail the script; the visual verdict —
**themed desktop, no autogenerated-config banner, no login prompt** — is
yours, from `first-boot.png`. The helpers it builds on live in
`tools/vm/` (QMP key injection, GL-safe VNC screenshots) and work for any
manual VM poking; `tools/vm/gap-analysis.py` is the maintainer tool for
diagnosing offline installs that try to build from source.
To test by hand instead, replicate what `tools/test-install.sh` does: the
unattended env it uses is in the script, and the same flow works
interactively from the live terminal. If the desktop comes up unthemed,
read `/var/log/nomarchy-hm-preactivate.log` on the installed system.
## 5. VM-specific gotchas
- **No KVM** (e.g. nested without acceleration): everything works but
`home-manager switch` will be painfully slow. Don't read slowness as
breakage.
- **Black screen / session exits**: almost always missing guest GL. Use the
script's qemu flags; on real hardware check `journalctl -b -u greetd`.
- **Tiny resolution**: the live config forces `monitor = ,highres,auto,1`;
if QEMU still picks 1024×768, resize the window — Hyprland follows.
- **First theme switch is the slowest**: it evaluates the flake on a RAM
disk. Subsequent switches reuse the eval cache.
## 6. When something fails
- Session/login problems: `journalctl -b -u greetd`, then `journalctl
--user -b`.
- Theme switch failures: run `nomarchy-theme-sync apply <x>` from a
terminal — the home-manager output streams there. The state file is
written *before* the rebuild, so after fixing you can just re-run
`home-manager switch --flake ~/.nomarchy`.
- Report results faithfully: what booted, what rendered, what you could
not verify and why.