Bernardo Magri
c9dd7575bf
Some checks failed
Check / eval (push) Has been cancelled
RECOVERY §4 documents the 14-day / keep-≥3-past policy for system and Home Manager gens, with dry-run and timer commands. README and REQUIREMENTS point here so install and day-2 hygiene match the product.
134 lines
5.6 KiB
Markdown
134 lines
5.6 KiB
Markdown
# Recovery runbook — when something breaks
|
||
|
||
Ordered from "the desktop looks wrong" to "the machine won't boot".
|
||
Everything here already ships on an installed machine — you don't need a
|
||
live USB until the last resort. The theme is always the same: **nothing
|
||
in Nomarchy is destroyed by a bad change** — every rebuild is a NixOS /
|
||
Home Manager generation you can step back to, and (on BTRFS installs)
|
||
snapper keeps file-level history on top.
|
||
|
||
If the graphical session is unusable, a text console is one keystroke
|
||
away: **Ctrl+Alt+F2** gives a TTY login (the session itself runs on
|
||
tty1); log in with your normal user.
|
||
|
||
## 1. A theme or desktop change broke the session
|
||
|
||
Theme applies and `home-update` are Home Manager switches — one
|
||
generation each, so the previous desktop is still on disk:
|
||
|
||
```sh
|
||
home-manager generations # newest first, one per theme/HM change
|
||
/nix/store/…-home-manager-generation/activate # run the one you want
|
||
```
|
||
|
||
The same picker lives in the menu: **System › Recovery › Desktop
|
||
generation** lists the recent Home Manager generations — pick one and
|
||
it activates. System-level undo is **System › Recovery › System boot
|
||
generation** (boot menu) or **Files (BTRFS)** (snapper).
|
||
|
||
Or simply apply a theme you know is good: `nomarchy-state-sync apply
|
||
boreal` (or any preset). If a switch failed halfway, the state file is written
|
||
*before* the rebuild — fix the cause and re-run
|
||
`home-manager switch --flake ~/.nomarchy` (or `nomarchy-home`).
|
||
|
||
Your flake checkout is a git repo, and with auto-commit enabled every
|
||
apply is a commit — and every `nomarchy-pull`/`-rebuild`/`-home` first
|
||
sweeps pending hand edits into one, so history mirrors your generations:
|
||
`git -C ~/.nomarchy log` to see what changed, `git revert` the culprit,
|
||
then `nomarchy-home`.
|
||
|
||
## 2. The desktop won't start at all
|
||
|
||
Greeter loops, black screen after the password, session exits straight
|
||
back to tuigreet — from the Ctrl+Alt+F2 TTY:
|
||
|
||
```sh
|
||
journalctl -b -u greetd # did the session command launch?
|
||
journalctl --user -b # Hyprland + the user services
|
||
```
|
||
|
||
- Rolling back the *desktop* half is §1 (works from the TTY).
|
||
- If greetd/tuigreet itself is broken, that's system-side → §3.
|
||
- In a **VM**, a black screen is almost always missing guest OpenGL,
|
||
not your config — see docs/TESTING.md §5.
|
||
- First boot after an install came up *unthemed*: read
|
||
`/var/log/nomarchy-hm-preactivate.log` on the installed system.
|
||
|
||
## 3. A system change broke it — boot an older generation
|
||
|
||
Reboot and pick an older **NixOS generation** in the systemd-boot menu
|
||
(hold a key during firmware handoff if the menu doesn't linger). That
|
||
boots yesterday's system unchanged.
|
||
|
||
Booting an old generation is temporary — the default entry is still the
|
||
broken one. Make the fix stick from the working boot: revert the change
|
||
in `~/.nomarchy` (`git -C ~/.nomarchy revert …` or edit `system.nix`
|
||
back), then `nomarchy-rebuild`.
|
||
|
||
How long generations stick around (and how cleanup works) is §4 below.
|
||
|
||
## 4. How generations are kept (and cleaned up)
|
||
|
||
Every `nomarchy-rebuild` and `nomarchy-home` (and every theme switch) leaves
|
||
a **generation** — a full previous system or desktop you can roll back to
|
||
(§1 and §3, and **System › Recovery**). Those take disk space until they
|
||
are removed.
|
||
|
||
Nomarchy prunes them **automatically once a week** with a policy designed
|
||
not to strand you without rollback:
|
||
|
||
| Rule | Meaning |
|
||
|------|---------|
|
||
| **Age** | Generations older than **14 days** are eligible for removal. |
|
||
| **Safety floor** | Always keep the **current** generation and at least **3 past** ones — even if those three are older than 14 days. |
|
||
| **Both halves** | The same rule applies independently to the **system** (boot menu) and to **Home Manager** (desktop) profiles. |
|
||
|
||
So a busy machine that rebuilds often can still free old junk after two
|
||
weeks, while a rarely rebuilt laptop never drops below four usable
|
||
system gens (current + three past) or four desktop gens.
|
||
|
||
**What you do not need to do:** run aggressive `nix-collect-garbage -d`
|
||
day to day — that can delete *all* old generations and wipe the floor.
|
||
The weekly timer already reclaims store paths after a careful prune.
|
||
|
||
**Manual control:**
|
||
|
||
```sh
|
||
sudo nomarchy-gen-prune --dry-run # list what would be deleted
|
||
sudo nomarchy-gen-prune # prune now (same rules as the timer)
|
||
systemctl status nomarchy-gen-prune.timer
|
||
```
|
||
|
||
After a system prune the boot menu is refreshed so removed generations
|
||
no longer appear as boot entries.
|
||
|
||
## 5. Files went missing or wrong — snapshots (BTRFS installs)
|
||
|
||
With `nomarchy.system.snapper.enable` (the installer's default on
|
||
BTRFS), the root filesystem has hourly/daily history, and
|
||
`nixos-rebuild-snap` leaves a snapshot right before a rebuild:
|
||
|
||
- **GUI:** menu › System › Recovery › Files (BTRFS) (btrfs-assistant;
|
||
expects a polkit password prompt).
|
||
- **Terminal/SSH:** `sudo nomarchy-snapshots` — browse a snapshot's
|
||
diff, **restore changed files** (snapper `undochange`), or **roll the
|
||
whole root back** to a snapshot and reboot. Both destructive actions
|
||
sit behind a typed-`yes` confirmation.
|
||
|
||
Snapshots are the undo for *data on disk*; the Nix config model is
|
||
undone by generations (§1/§3) — use each for its half.
|
||
|
||
## 5. Last resort — from the outside
|
||
|
||
Boot the Nomarchy ISO (any NixOS ISO works), then:
|
||
|
||
```sh
|
||
sudo mount /dev/<root> /mnt # + /mnt/boot; LUKS: cryptsetup open first
|
||
sudo nixos-enter --root /mnt # chroot with nix available
|
||
```
|
||
|
||
From there you have the full toolbox: `nixos-rebuild boot --flake
|
||
/home/<you>/.nomarchy#default` after fixing the flake, or snapper from
|
||
§4. If you get this far with something Nomarchy shipped broken, please
|
||
file it.
|