Nomarchy/docs/RECOVERY.md

# Recovery runbook — when something breaks

Ordered from "the desktop looks wrong" to "the machine won't boot".
Everything here already ships on an installed machine — you don't need a
live USB until the last resort. The theme is always the same: **nothing
in Nomarchy is destroyed by a bad change** — every rebuild is a NixOS /
Home Manager generation you can step back to, and (on BTRFS installs)
snapper keeps file-level history on top.

If the graphical session is unusable, a text console is one keystroke
away: **Ctrl+Alt+F2** gives a TTY login (the session itself runs on
tty1); log in with your normal user.

## 1. A theme or desktop change broke the session

Theme applies and `home-update` are Home Manager switches — one
generation each, so the previous desktop is still on disk:

```sh
home-manager generations          # newest first, one per theme/HM change
/nix/store/…-home-manager-generation/activate   # run the one you want
```

The same picker lives in the menu: **System › Recovery › Desktop
generation** lists the recent Home Manager generations — pick one and
it activates. System-level undo is **System › Recovery › System boot
generation** (boot menu) or **Files (BTRFS)** (snapper).

Or simply apply a theme you know is good: `nomarchy-state-sync apply
boreal` (or any preset). If a switch failed halfway, the state file is written
*before* the rebuild — fix the cause and re-run
`home-manager switch --flake ~/.nomarchy` (or `nomarchy-home`).

Your flake checkout is a git repo, and with auto-commit enabled every
apply is a commit — and every `nomarchy-pull`/`-rebuild`/`-home` first
sweeps pending hand edits into one, so history mirrors your generations:
`git -C ~/.nomarchy log` to see what changed, `git revert` the culprit,
then `nomarchy-home`.

## 2. The desktop won't start at all

Greeter loops, black screen after the password, session exits straight
back to tuigreet — from the Ctrl+Alt+F2 TTY:

```sh
journalctl -b -u greetd           # did the session command launch?
journalctl --user -b              # Hyprland + the user services
```

- Rolling back the *desktop* half is §1 (works from the TTY).
- If greetd/tuigreet itself is broken, that's system-side → §3.
- Password rejected on an external keyboard but definitely correct? The
  greeter uses the system keyboard layout, not a remembered per-device one
  — see the "Greeter keyboard layout" note in README.md § options.
- In a **VM**, a black screen is almost always missing guest OpenGL,
  not your config — see docs/TESTING.md §5.
- First boot after an install came up *unthemed*: read
  `/var/log/nomarchy-hm-preactivate.log` on the installed system.

## 3. A system change broke it — boot an older generation

Reboot and pick an older **NixOS generation** in the systemd-boot menu
(hold a key during firmware handoff if the menu doesn't linger). That
boots yesterday's system unchanged.

Booting an old generation is temporary — the default entry is still the
broken one. Make the fix stick from the working boot: revert the change
in `~/.nomarchy` (`git -C ~/.nomarchy revert …` or edit `system.nix`
back), then `nomarchy-rebuild`.

How long generations stick around (and how cleanup works) is §4 below.

## 4. How generations are kept (and cleaned up)

Every `nomarchy-rebuild` and `nomarchy-home` (and every theme switch) leaves
a **generation** — a full previous system or desktop you can roll back to
(§1 and §3, and **System › Recovery**). Those take disk space until they
are removed.

Nomarchy prunes them **automatically once a week** with a policy designed
not to strand you without rollback:

| Rule | Meaning |
|------|---------|
| **Age** | Generations older than **14 days** are eligible for removal. |
| **Safety floor** | Always keep the **current** generation and at least **3 past** ones — even if those three are older than 14 days. |
| **Both halves** | The same rule applies independently to the **system** (boot menu) and to **Home Manager** (desktop) profiles. |

So a busy machine that rebuilds often can still free old junk after two
weeks, while a rarely rebuilt laptop never drops below four usable
system gens (current + three past) or four desktop gens.

**What you do not need to do:** run aggressive `nix-collect-garbage -d`
day to day — that can delete *all* old generations and wipe the floor.
The weekly timer already reclaims store paths after a careful prune.

**Manual control:**

```sh
sudo nomarchy-gen-prune --dry-run   # list what would be deleted
sudo nomarchy-gen-prune             # prune now (same rules as the timer)
systemctl status nomarchy-gen-prune.timer
```

After a system prune the boot menu is refreshed so removed generations
no longer appear as boot entries.

## 5. Files went missing or wrong — snapshots (BTRFS installs)

With `nomarchy.system.snapper.enable` (the installer's default on
BTRFS), the root filesystem has hourly/daily history, and
`nixos-rebuild-snap` leaves a snapshot right before a rebuild:

- **GUI:** menu › System › Recovery › Files (BTRFS) (btrfs-assistant;
  expects a polkit password prompt).
- **Terminal/SSH:** `sudo nomarchy-snapshots` — browse a snapshot's
  diff, **restore changed files** (snapper `undochange`), or **roll the
  whole root back** to a snapshot and reboot. Both destructive actions
  sit behind a typed-`yes` confirmation.

Snapshots are the undo for *data on disk*; the Nix config model is
undone by generations (§1/§3) — use each for its half.

## 6. Last resort — from the outside

Boot the Nomarchy ISO (any NixOS ISO works), then:

```sh
sudo mount /dev/<root> /mnt        # + /mnt/boot; LUKS: cryptsetup open first
sudo nixos-enter --root /mnt       # chroot with nix available
```

From there you have the full toolbox: `nixos-rebuild boot --flake
/home/<you>/.nomarchy#default` after fixing the flake, or snapper from
§5. If you get this far with something Nomarchy shipped broken, please
file it.