docs(recovery): the undo runbook — generations, journals, snapper, nixos-enter (item 6d)
All checks were successful
Check / eval (push) Successful in 3m2s

Slice (d) of the docs review: docs/RECOVERY.md tells the recovery
story that existed only in pieces, ordered by symptom — bad theme/HM
change (generations, apply-known-good, git-revert the flake), desktop
won't start (TTY, greetd + user journals, VM-GL and preactivate-log
pointers), broken system change (older systemd-boot generation, then
make it stick), lost files (btrfs-assistant / nomarchy-snapshots), and
the ISO + nixos-enter last resort. Commands checked against the tree;
linked from the README file tree and §5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Bernardo Magri
2026-07-04 19:41:05 +01:00
parent b0b8a9a09b
commit 472d7502b4
4 changed files with 116 additions and 7 deletions

View File

@@ -76,6 +76,7 @@ Flat on purpose. Two module trees, one options file each, no hidden layers.
├── templates/downstream/ # `nix flake init -t` starter for users
├── docs/TESTING.md # how to verify changes (incl. AI-agent rules)
├── docs/OVERRIDES.md # how downstream users override defaults
├── docs/RECOVERY.md # runbook: broken theme/desktop/boot → undo it
├── docs/ROADMAP.md # design/decision records + shipped-fixes log
├── agent/ # autonomous-agent loop: protocol (LOOP.md),
│ # prioritized BACKLOG, journal, memory
@@ -361,6 +362,10 @@ sys-rebuild # rebuild the system, current lock (no up
home-update # rebuild just the desktop layer
```
Something broke anyway? Every rebuild is a generation and (on BTRFS)
every hour is a snapshot — the undo story, from a bad theme to a
machine that won't boot, is **[docs/RECOVERY.md](docs/RECOVERY.md)**.
Keybinds: `SUPER+Return` terminal · `SUPER+D` launcher · `SUPER+T` theme
picker · `SUPER+SHIFT+T` next wallpaper · `SUPER+X` power menu ·
`SUPER+E` file manager (yazi) · `SUPER+N` notifications · `SUPER+CTRL+V`

View File

@@ -30,14 +30,14 @@ surface vs tables, catches stale rows too); slice (b) drift pass over
docs/OVERRIDES.md, docs/TESTING.md, templates/downstream/README.md —
every factual claim spot-checked against the tree, all held; fixed the
two gaps (OVERRIDES now routes to `nomarchy.monitors`; TESTING §1 lists
the shell-syntax check its CI § claimed). Remaining slices:
the shell-syntax check its CI § claimed); slice (d) recovery runbook —
docs/RECOVERY.md (theme/HM generations → greetd journals → boot an older
generation → snapper restore/rollback → nixos-enter), linked from the
README tree + §5. Remaining:
- **(c) install/first-run story** read end to end (README install §§ +
template README as one narrative; fix the seams).
- **(d) recovery runbook** — short doc: desktop won't start → boot an
older generation / `nomarchy-snapshots` / greetd journal. The pieces
exist, the story isn't written.
Site-vs-markdown is a Decision (below). **Verify:** V0 per slice; (b)/(c)
against the live tree, not memory.
template README as one narrative; fix the seams) — the last slice.
Site-vs-markdown is a Decision (below). **Verify:** V0; (c) against the
live tree, not memory.
## NEXT

View File

@@ -17,6 +17,21 @@ Template:
---
## 2026-07-04 — docs slice (d): recovery runbook (iteration #15, item 6d)
- **Task:** BACKLOG NOW#6 slice (d) — write the undo story.
- **Did:** docs/RECOVERY.md, symptom-ordered: (1) bad theme/HM change →
generations / apply-known-good / git-revert ~/.nomarchy; (2) desktop
won't start → Ctrl+Alt+F2, greetd + user journals, VM-GL and
preactivate-log pointers; (3) system change → systemd-boot older
generation (kept: 10) + make-it-stick via sys-rebuild; (4) files →
btrfs-assistant / sudo nomarchy-snapshots (undochange, root rollback,
typed-yes); (5) ISO + nixos-enter. Every command checked against the
tree (greetd cfg, configurationLimit, snapshots script actions,
helper names). Linked from README tree + §5.
- **Verified:** V0 (docs-only) — flake check clean pre-commit.
- **Pending:** slice (c) install-story read-through — the last of item 6.
- **Next suggestion:** finish item 6 with slice (c).
## 2026-07-04 — docs slice (b): drift pass (iteration #14, item 6b)
- **Task:** BACKLOG NOW#6 slice (b) — OVERRIDES.md / TESTING.md /
templates/downstream/README.md vs the live tree.

89
docs/RECOVERY.md Normal file
View File

@@ -0,0 +1,89 @@
# Recovery runbook — when something breaks
Ordered from "the desktop looks wrong" to "the machine won't boot".
Everything here already ships on an installed machine — you don't need a
live USB until the last resort. The theme is always the same: **nothing
in Nomarchy is destroyed by a bad change** — every rebuild is a NixOS /
Home Manager generation you can step back to, and (on BTRFS installs)
snapper keeps file-level history on top.
If the graphical session is unusable, a text console is one keystroke
away: **Ctrl+Alt+F2** gives a TTY login (the session itself runs on
tty1); log in with your normal user.
## 1. A theme or desktop change broke the session
Theme applies and `home-update` are Home Manager switches — one
generation each, so the previous desktop is still on disk:
```sh
home-manager generations # newest first, one per theme/HM change
/nix/store/…-home-manager-generation/activate # run the one you want
```
Or simply apply a theme you know is good: `nomarchy-theme-sync apply
tokyo-night`. If a switch failed halfway, the state file is written
*before* the rebuild — fix the cause and re-run
`home-manager switch --flake ~/.nomarchy` (or `home-update`).
Your flake checkout is a git repo, and with auto-commit enabled every
apply is a commit: `git -C ~/.nomarchy log` to see what changed,
`git revert` the culprit, then `home-update`.
## 2. The desktop won't start at all
Greeter loops, black screen after the password, session exits straight
back to tuigreet — from the Ctrl+Alt+F2 TTY:
```sh
journalctl -b -u greetd # did the session command launch?
journalctl --user -b # Hyprland + the user services
```
- Rolling back the *desktop* half is §1 (works from the TTY).
- If greetd/tuigreet itself is broken, that's system-side → §3.
- In a **VM**, a black screen is almost always missing guest OpenGL,
not your config — see docs/TESTING.md §5.
- First boot after an install came up *unthemed*: read
`/var/log/nomarchy-hm-preactivate.log` on the installed system.
## 3. A system change broke it — boot an older generation
Reboot and pick an older **NixOS generation** in the systemd-boot menu
(hold a key during firmware handoff if the menu doesn't linger; the
last 10 generations are kept). That boots yesterday's system unchanged.
Booting an old generation is temporary — the default entry is still the
broken one. Make the fix stick from the working boot: revert the change
in `~/.nomarchy` (`git -C ~/.nomarchy revert …` or edit `system.nix`
back), then `sys-rebuild`.
## 4. Files went missing or wrong — snapshots (BTRFS installs)
With `nomarchy.system.snapper.enable` (the installer's default on
BTRFS), the root filesystem has hourly/daily history, and
`nixos-rebuild-snap` leaves a snapshot right before a rebuild:
- **GUI:** menu System Snapshots (btrfs-assistant; expects a polkit
password prompt).
- **Terminal/SSH:** `sudo nomarchy-snapshots` — browse a snapshot's
diff, **restore changed files** (snapper `undochange`), or **roll the
whole root back** to a snapshot and reboot. Both destructive actions
sit behind a typed-`yes` confirmation.
Snapshots are the undo for *data on disk*; the Nix config model is
undone by generations (§1/§3) — use each for its half.
## 5. Last resort — from the outside
Boot the Nomarchy ISO (any NixOS ISO works), then:
```sh
sudo mount /dev/<root> /mnt # + /mnt/boot; LUKS: cryptsetup open first
sudo nixos-enter --root /mnt # chroot with nix available
```
From there you have the full toolbox: `nixos-rebuild boot --flake
/home/<you>/.nomarchy#default` after fixing the flake, or snapper from
§4. If you get this far with something Nomarchy shipped broken, please
file it.