Files
Nomarchy/docs/TESTING.md
Bernardo Magri e6be5a5770
Some checks failed
Check / eval (push) Has been cancelled
fix: disk picker fd0 (#112), menu Back guard (#106), offline theme (#113)
- #112: list_installable_disks drops fd/loop/sr/zram/<8GiB, largest-first
  so OVMF no longer offers /dev/fd0 as the default wipe target
- #106: tools/check-menu-back.py + checks.menu-back (proved to fail)
- #113: offline theme-switch contract documented (default/pinned only);
  theme-sync run_switch adds offline-oriented hint when no network

Verified: V0 flake check; installer-safety; menu-back; py_compile.
2026-07-15 10:02:19 +01:00

146 lines
7.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Testing Nomarchy
How to verify changes actually work — from cheap eval checks to booting the
full desktop. Adapted from the previous Nomarchy iteration's agent docs.
**The honesty rule (for humans and AI agents alike):** for Waybar, Hyprland,
or any visual change, the only reliable check is booting the live ISO. If
you can't boot it (e.g. you're on macOS or have no KVM), **say so** rather
than claiming success. "All Nix files parse" is not "the bar renders."
## 1. Cheap checks (always do these first)
```sh
nix flake check --no-build # full module-system evaluation, no builds
nix-instantiate --parse <file> # syntax-only, works even on macOS
git ls-files '*.py' | xargs python3 -m py_compile # all tracked Python
bash -n <script>.sh # shell syntax (tools/, installer bits)
```
`nix flake check` needs a Linux machine (or a Linux builder) since all
outputs are `x86_64-linux`. It catches type errors, missing options, and
bad merges — most breakage stops here. It also evaluates the downstream
template through `lib.mkFlake` (including a real nixos-hardware profile),
so template/wrapper drift fails fast too.
## 1b. CI (automatic on push)
Every push to `main`/`v1` runs `.gitea/workflows/check.yml`: the §1
cheap checks (flake eval, Python + shell syntax) on the Gitea instance.
That's the **eval tier only** — the runner is a docker container without
KVM, so the `checks.*` VM suite and real builds stay local (this file)
until a KVM-capable runner is registered; the workflow carries a
commented `vm-checks` job ready for that day. A green CI run is *not* "it
renders" (the honesty rule below still applies) — it means "nobody broke
evaluation".
## 2. Build and boot the live ISO
```sh
# Build + boot in QEMU, one command:
tools/test-live-iso.sh
# Or by hand:
nix build .#nixosConfigurations.nomarchy-live.config.system.build.isoImage
# → result/iso/*.iso — dd it to a USB stick for real hardware
```
The script prefers UEFI (OVMF) with a legacy-BIOS fallback, uses KVM when
`/dev/kvm` is readable, and boots with `virtio-vga-gl` + `gl=on`
**Hyprland (and the desktop session) need real OpenGL in the guest**; without it the
session may not start.
### What the live environment gives you
- Auto-login as `nomarchy` (no password, passwordless sudo), greetd →
Hyprland via `initial_session`; logging out lands on tuigreet.
- The flake is seeded writable at `~/.nomarchy` (from the read-only
`/etc/nomarchy` copy) — `$NOMARCHY_PATH` already points there.
- Locked flake inputs are pinned into the ISO store, so a
`home-manager switch` for the **already-active / default theme** works
**without a network**. Switching to an arbitrary other preset can still
need downloads (that preset's HM generation is not in the pin) — the
tool should fail with an offline-oriented message (#113).
## 3. Verification checklist
Work through these in order; each one exercises a different layer.
| # | Check | Verifies |
|---|---|---|
| 1 | Boots to Hyprland with the default theme wallpaper visible (Boreal) | greetd autologin, awww, session start |
| 2 | Waybar shows at top, themed (blue accent on dark) | HM waybar module, palette baking |
| 3 | `SUPER+Return` opens Kitty with the default theme colors (Boreal) | terminal default, ANSI palette |
| 4 | `btop` in the terminal is themed | per-theme asset baking |
| 5 | `nomarchy-theme-sync list` prints 24 presets | package, baked themes dir |
| 6 | `nomarchy-theme-sync apply boreal` (or re-apply the **current** theme) offline → switch succeeds. Applying a *different* preset may need a network — the ISO pins the default theme's generation, not every preset (#113) | state write + offline rebuild contract |
| 7 | `SUPER+SHIFT+T` cycles wallpapers instantly (try `tokyo-night` for 4, or `boreal` for its set) | the runtime wallpaper path |
| 8 | `nomarchy-theme-sync apply summer-night` → after the switch the bar has its own identity (light bar, different styling) | whole-swap waybar.css assets |
| 9 | Open a GTK app — dark theme matching the palette | Stylix layer |
| 10 | `home-manager generations` lists one generation per theme change; activating an older one rolls the theme back | atomicity / rollback story |
Items 6 and 8 are the big ones — they prove the all-Home-Manager theming
model end to end.
## 4. Testing the installer
One command runs the whole offline regression:
```sh
tools/test-install.sh
```
It builds the ISO, boots it in QEMU **with networking disabled**, runs an
unattended LUKS+swap install onto a blank disk via the installer's
`NOMARCHY_UNATTENDED=1` mode (config delivered on a small vfat disk —
typing long commands into a guest drops keystrokes), waits for the
installer's poweroff, then boots the installed disk, enters the
passphrase, and screenshots the first boot. The machine-checkable parts
(install completes, disk boots) fail the script; the visual verdict —
**themed desktop, no autogenerated-config banner, no login prompt** — is
yours, from `first-boot.png`. The helpers it builds on live in
`tools/vm/` (QMP key injection, GL-safe VNC screenshots) and work for any
manual VM poking; `tools/vm/gap-analysis.py` is the maintainer tool for
diagnosing offline installs that try to build from source.
To test by hand instead, replicate what `tools/test-install.sh` does: the
unattended env it uses is in the script, and the same flow works
interactively from the live terminal. If the desktop comes up unthemed,
read `/var/log/nomarchy-hm-preactivate.log` on the installed system.
Two full-desktop VM harnesses (softGL Hyprland, too heavy for `checks.*`)
live next to the scripts: `tools/theme-shot.nix` (themed-desktop
screenshots) and `tools/monitor-fallback.nix` (KVM display transitions:
atomic dock handoff, internal-first undock, and low-level lid-inhibitor
lifecycle). The latter restores the DRM output before deleting QEMU's final
synthetic headless output; physical cable-removal timing is a V3 check because
Hyprland's headless backend otherwise aborts in a VM-only zero-output state.
## 5. VM-specific gotchas
- **No KVM** (e.g. nested without acceleration): everything works but
`home-manager switch` will be painfully slow. Don't read slowness as
breakage.
- **Black screen / session exits**: almost always missing guest GL. Use the
script's qemu flags; on real hardware check `journalctl -b -u greetd`.
- **Tiny resolution**: the live config forces `monitor = ,highres,auto,1`;
if QEMU still picks 1024×768, resize the window — Hyprland follows.
- **Display menu + QEMU**: System Display mode list is sorted by **pixel
area** (highest first). After a pick, the toast shows hyprctls reported
size. In a **fixed-size** QEMU/viewer window, a higher guest mode still
*looks* smaller (more pixels in the same window) — that is viewer scaling,
not an inverted apply. Trust hyprctl / the toast numbers.
- **First theme switch is the slowest**: it evaluates the flake on a RAM
disk. Subsequent switches reuse the eval cache.
## 6. When something fails
- Session/login problems: `journalctl -b -u greetd`, then `journalctl
--user -b`.
- Theme switch failures: run `nomarchy-theme-sync apply <x>` from a
terminal — the home-manager output streams there. The state file is
written *before* the rebuild, so after fixing you can just re-run
`home-manager switch --flake ~/.nomarchy`.
- Report results faithfully: what booted, what rendered, what you could
not verify and why.