From e324876eb784a9956be45c20dcff19e13f09d074 Mon Sep 17 00:00:00 2001 From: Bernardo Magri Date: Fri, 17 Jul 2026 07:44:43 +0100 Subject: [PATCH] =?UTF-8?q?feat(doctor):=20#147=20=E2=80=94=20flag=20machi?= =?UTF-8?q?ne=20state=20the=20flake=20never=20set;=20linger=20is=20the=20w?= =?UTF-8?q?orked=20example?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit /var/lib survives a migration untouched, and a stray pre-Nomarchy Linger=yes marker shaped the QA box's session behavior for six months with nothing in the flake to show for it. Ship the class as guards: - nomarchy-doctor: new machine-state-drift check after the flake-checkout check — loginctl show-user -p Linger; "yes" warns (nothing is broken — it is drift) and prints the disable-linger fix; unknown-to-logind skips. - docs/MIGRATION.md § Post-migration cleanup: "/var/lib machine state survives migration" — the worked example and the general suspicion rule. - checks.doctor: linger sub-case — warn fires under enable-linger, exit stays 0, warning clears after disable-linger. Deliberately NOT hardening the session wiring against the portal-gtk restart race linger enabled: a clean install cannot reach that state. The Waybar exec-once loose thread survives as BACKLOG #149. Verification: V2 — shellcheck clean, nix flake check --no-build green, nix build .#checks.x86_64-linux.doctor green (both branches exercised). Implementation by a Sonnet subagent; design and review on Fable. Co-Authored-By: Claude Fable 5 --- agent/BACKLOG.md | 66 ++++--------------------- agent/JOURNAL.md | 15 ++++++ docs/MIGRATION.md | 20 ++++++++ docs/ROADMAP.md | 11 +++++ flake.nix | 12 +++++ pkgs/nomarchy-doctor/nomarchy-doctor.sh | 13 +++++ 6 files changed, 80 insertions(+), 57 deletions(-) diff --git a/agent/BACKLOG.md b/agent/BACKLOG.md index da87310..b8cb865 100644 --- a/agent/BACKLOG.md +++ b/agent/BACKLOG.md @@ -265,64 +265,16 @@ If that count is zero over a few weeks post-linger, **close this unbuilt**. If i is not, the upstream issue has sat ten months on vaxerski's "that will cause a segfault no?" and a reproducer would be worth more than a local guard. -### 147. Nomarchy cannot see machine state it never set — and it bit the QA box for 6 months +### 149. `[watch]` Waybar's exec-once workaround may be defending against nothing post-linger -**Rewritten 2026-07-16 after diagnosis; the original filing was wrong and its -wrongness is the point.** It claimed "a re-login leaves session units dead; -the repo never adopted the standard session wiring". The repo *has* the wiring -— HM's `wayland.windowManager.hyprland.systemd.enable` defaults on and is line -1 of the generated `hyprland.conf` — and it fired correctly. Do not go add it. - -**What actually happened**, from systemd's own words at the 17:25:54 relogin: - -``` -Stopped target Hyprland compositor session. -Requested transaction contradicts existing jobs: Transaction for -graphical-session.target/stop is destructive (xdg-desktop-portal-gtk.service -has 'start' job queued, but 'stop' is included in transaction). -graphical-session.target: Failed to enqueue stop job, ignoring -Reached target Hyprland compositor session. -``` - -`hyprland-session.target` stopped; `StopWhenUnneeded=yes` correctly tried to -take `graphical-session.target` with it; systemd **refused the stop** because -portal-gtk had a `start` job queued from its own post-crash restart loop (3 -restart/fail lines in that 13s gap). So graphical-session never went down, 8ms -later hyprland-session came back to find it active, and nothing `WantedBy` it -was re-pulled. cliphist had already burned 5 restarts into `start-limit-hit`; -nm-applet/udiskie have no restart policy and just stayed dead. - -**But none of that can happen on a Nomarchy install.** The whole sequence needs -the user manager to survive logout, i.e. `Linger=yes` — and linger came from an -empty marker in `/var/lib/systemd/linger/bernardo` dated **2026-01-24**, which -**pre-dates Bernardo's migration to Nomarchy** (he was on a custom flake then). -Nothing here sets it: not modules, not hosts, not templates, and `git log -S` -says it was never shipped, ever. Removed 2026-07-16 (`loginctl disable-linger`, -no restart needed); nothing wanted it — `nomarchy-updates.timer` is -`Persistent=yes` and catches up on login. - -**So the real item is the class, not the bug.** `/var/lib` is machine state; a -migration reconciles the flake and leaves everything else exactly as it was. -Nomarchy therefore ran for six months on its own primary V3 box with a setting -it never chose, never set, and could not see — on the machine whose whole job is -telling us how the distro behaves. That is a direct hit on the in-flake-state -promise (state outside the checkout) *and* on V3's trustworthiness: a QA box -that silently diverges reports on a distro nobody ships. - -**Pitch:** a `nomarchy-doctor` check for drift of this class — linger is the -worked example (`loginctl show-user $USER -p Linger` vs. what the flake asks -for), and MIGRATION.md should say plainly that `/var/lib` survives and what to -sweep. Cost is small and it is the only part of this worth shipping. **Do not** -harden the session wiring against the portal-gtk race: a clean install cannot -reach it, and defending against it would be clever code for a state we do not -ship. - -**Loose thread for whoever takes this:** `modules/home/hyprland.nix` runs Waybar -from `exec-once` rather than a unit because the unit "raced Hyprland's IPC on a -**warm** relogin ... never retried, so the bar vanished" — and a warm relogin is -exactly what linger creates. That workaround may now be defending against -nothing. Do not rip it out on this note alone; it works, and the claim needs a -real relogin without linger to test. +Spun out of #147 (shipped 2026-07-17) so its loose thread survives the entry: +`modules/home/hyprland.nix` runs Waybar from `exec-once` rather than a systemd +unit because the unit "raced Hyprland's IPC on a **warm** relogin ... never +retried, so the bar vanished" — and a warm relogin is exactly what linger +created. With the stray linger removed (2026-07-16) that workaround may be +defending against nothing. Do not rip it out on this note alone: it works, and +the claim needs a real relogin without linger to test (V3 — the same relogin +that verifies #148's watcher fix would answer it). ### 120. A netinstall ISO, next to the fat offline one diff --git a/agent/JOURNAL.md b/agent/JOURNAL.md index 81dfb52..ff911e6 100644 --- a/agent/JOURNAL.md +++ b/agent/JOURNAL.md @@ -19,6 +19,21 @@ Template: --- +## 2026-07-17 — #147: doctor learns to see machine state the flake never set +- **Task:** BACKLOG #147 (promoted from PROPOSED by Bernardo in-session). +- **Did:** `nomarchy-doctor` gains a machine-state-drift check — linger as the + worked example (`loginctl show-user -p Linger`; `yes` → warn + fix command, + unknown-to-logind → skip); MIGRATION.md § Post-migration cleanup gains + "`/var/lib` machine state survives migration"; `checks.doctor` grew the + warn-fires/warn-clears linger sub-case. Implementation delegated to a + Sonnet agent, design + review here. Loose thread preserved as #149. +- **Verified:** V2 — shellcheck clean, `nix flake check --no-build` green, + `nix build .#checks.x86_64-linux.doctor` green (linger sub-case exercised + both branches, doctor exit 0 throughout). +- **Pending:** nothing hardware-blocked; the real-world case (QA box linger) + was already fixed by hand 2026-07-16. +- **Next suggestion:** hardware hints, then #148 review. + ## 2026-07-17 — Live triage with Bernardo: #114 ships as docs; rulings on #120/#134; #143 checked - **Task:** Bernardo triaged PROPOSED in-session: #148/#147/hardware-hints promoted (in flight as separate commits), #120 re-affirmed deferred, #134 diff --git a/docs/MIGRATION.md b/docs/MIGRATION.md index 152cd50..580039d 100644 --- a/docs/MIGRATION.md +++ b/docs/MIGRATION.md @@ -384,6 +384,26 @@ lock). prototype). - Prune old generations: `sudo nix-collect-garbage -d`. +### `/var/lib` machine state survives migration + +A migration reconciles the **flake** — packages, services, dotfiles — and +nothing else. Everything under `/var/lib` is left exactly as the previous +OS wrote it, including settings the flake can neither see nor express. That +state doesn't show up in `git diff`, doesn't get touched by `sys-rebuild`, +and can sit there for months quietly shaping how the machine behaves. + +Worked example: user lingering. Run `loginctl show-user $USER -p Linger` — +on a Nomarchy machine the answer should be `no`, because the flake never +sets it. On our own QA box a leftover `Linger=yes` from a pre‑Nomarchy setup +kept the user's systemd instance alive across logouts, and it took six +months to trace a string of subtle session bugs back to that one marker. +Fix: `sudo loginctl disable-linger `. + +`nomarchy-doctor` now flags this specific case as a warning. More broadly: +after migrating, if the machine ever behaves in a way the flake can't +explain, suspect inherited state under `/var/lib` before you suspect the +flake. + --- ## Enabling hibernation on an existing machine (no reinstall) diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 3db0f4a..85b9254 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -493,6 +493,17 @@ Design/decision records and a running log of shipped work (items marked V2 by render, both heads + the password dialog, via the new `tools/plymouth-preview.sh`. V3 (a real docked boot/shutdown) is queued: the fake heads exist from the start, so the rig cannot resize a canvas. +- ✓ **Machine state the flake can't see (#147, 2026-07-17):** the QA box ran + six months with `Linger=yes` from a pre-migration marker in + `/var/lib/systemd/linger` — set by nothing in the repo, visible to no flake + audit. The class ships as guards, not session hardening: `nomarchy-doctor` + now warns on linger drift (`loginctl show-user -p Linger` vs. the flake's + implicit "no", fix command printed), and MIGRATION.md § Post-migration + cleanup states plainly that `/var/lib` survives a migration and what to + sweep. The portal-gtk restart race that linger enabled was deliberately NOT + hardened against — a clean install cannot reach that state. V2: + `checks.doctor` grew a linger sub-case (warn fires, exit stays 0, clears + after `disable-linger`). The Waybar exec-once loose thread became #149. - ✓ **Dock mode survives a rebuild (#142, 2026-07-16):** Bernardo: every rebuild while docked re-lit the laptop panel and threw a workspace onto it, and he re-picked Dock mode by hand. It was not forgetfulness — **a rebuild performed diff --git a/flake.nix b/flake.nix index a34b8d4..51544fb 100644 --- a/flake.nix +++ b/flake.nix @@ -1017,6 +1017,18 @@ healthy = machine.succeed("nomarchy-doctor 2>&1") for probe in ["NetworkManager", "fingerprint", "fwupd", "VA-API", "hibernate", "battery health"]: assert probe in healthy, f"doctor hardware section missing a self-gated '{probe}' row:\n{healthy}" + # Machine-state drift: linger (#147's worked example) — the + # flake never sets it, so a "yes" here is drift; doctor must + # warn (exit 0), not fail, and the warning must clear once the + # marker is gone. + machine.succeed("loginctl enable-linger root") + linger_on = machine.succeed("nomarchy-doctor 2>&1") + assert "Nomarchy never sets it" in linger_on, f"doctor did not flag linger drift:\n{linger_on}" + machine.succeed("loginctl disable-linger root") + linger_off = machine.succeed("nomarchy-doctor 2>&1") + assert "Nomarchy never sets it" not in linger_off, ( + f"doctor still flags linger drift after disable-linger:\n{linger_off}" + ) # #80 battery health: faked sysfs (test_power has no cycle/design). machine.succeed( "mkdir -p /tmp/fake-ps/BAT0 && " diff --git a/pkgs/nomarchy-doctor/nomarchy-doctor.sh b/pkgs/nomarchy-doctor/nomarchy-doctor.sh index 9757657..a1c58ab 100644 --- a/pkgs/nomarchy-doctor/nomarchy-doctor.sh +++ b/pkgs/nomarchy-doctor/nomarchy-doctor.sh @@ -110,6 +110,19 @@ if [ -d "$flake/.git" ]; then fi fi +# ── machine-state drift (survives migration, unseen by the flake) ──── +# Worked example: systemd linger. The flake never sets it anywhere, so +# a "yes" here is leftover state from a previous OS/install, not +# something Nomarchy configured — it can cause months of divergent +# session behavior that no amount of flake auditing will explain. +linger=$(loginctl show-user "${USER:-$(id -un)}" --property=Linger --value 2>/dev/null || true) +case "$linger" in + '') skip "user linger (user unknown to logind — no session)" ;; + yes) warn "user linger is enabled but Nomarchy never sets it (machine state the flake can't see)" \ + "loginctl disable-linger ${USER:-$(id -un)}" ;; + *) ok "no user linger (matches the flake)" ;; +esac + # ── generation age ─────────────────────────────────────────────────── # The profile SYMLINK's own mtime is the generation's creation time # (store paths themselves are all epoch-1).