All checks were successful
Check / eval (push) Successful in 3m41s
The incident was misdated: it did not happen before 12:51:34 Jul 15
(that boot was healthy all day) — it ended the previous boot at 10:54:22
with the power button. In that boot, running the pre-fix generation:
idle cascade on schedule (lock 10:21, dpms off 10:26, suspend correctly
skipped on AC), then hypridle logs "BUG THIS: inhibit locks < 0: -1" at
10:36:29 and its rule state is corrupted from there — the fired DPMS
listener's on-resume, then the session's ONLY dpms-on caller, never
dispatches again despite provable real input (two blind hyprlock PAM
attempts) and three output remove/re-add cycles. The wedged-compositor/
DRM-master lead is contradicted: Hyprland tracked hotplug live through
the whole window, PAM worked, kernel spotless. The shipped fix (a2151f4,
input wakes DPMS compositor-side) closes exactly this mechanism.
Filed upstream with the full trace: hyprwm/hypridle#208 (0.1.7; their
closed #104/#128 are the same class). Watch folded into #146.
Sweep: BACKLOG #127 deleted (NOW keeps only the install-findings context
stub, flagged to Bernardo); ROADMAP gains "The idle brick's CAUSE found"
and both stale "cause still open" tails patched; checked-off #127
HARDWARE-QUEUE blocks pruned, their method lessons moved to MEMORY
§ Gotchas.
Verification: V0 (docs/backlog only). The evidence is journal forensics
on the incident hardware (two Sonnet passes, verdict on Fable), quoted
in the ROADMAP entry; the fix itself was V3-proven 2026-07-16.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
14 KiB
14 KiB
Memory — durable lessons, learned the hard way
Curated, not append-only: one line per fact, newest at the top of its section; delete entries that stop being true. Details usually live in a docs/ROADMAP.md decision record — pointer given as (§ item). Add a fact here the moment a debugging session teaches you something a future iteration would otherwise rediscover.
Testing & VM recipes
- Doctor float V2:
THEME=<slug> nix build --impure -f tools/doctor-float.nix— softGL Hyprland,nomarchy-menu doctor, assertsclass=com.nomarchy.doctor+floating+ centered midpoints + screenshots (Kitty--class=…). SoftGL may still struggle with a GPU terminal; size can ignore percent windowrules if client geometry wins. - theme-shot softGL may not start Kitty —
btop.pngis best-effort (usually identical to desktop). Guest asserts on~/.config/btop/themes/nomarchy.themeprove baking; the TUI look is hardware/GL tier (HARDWARE-QUEUE). - Waybar
custom/doctortripwire: status helper must invokenomarchy-doctorby absolute store path (waybar's env can miss system PATH →command -v … || exit 0self-hides forever); empty"text"also self-hides; strip ANSI before packing the tooltip; use signal 10 +format = "{}". theme-shot asserts class:bad + glyph and pokes RTMIN+10 before the desktop shot. - tuigreet dies silently under runNixOSTest (even bare, no theme flag: greetd sits as "(greetd)" with no child, nothing in the journal — its stderr goes to the VT) — nixpkgs' own greetd test uses agreety instead. Greeter rendering is interactive-ISO/hardware tier; don't burn another session on a checks.greeter VM test.
- In VM tests
pgrep -f PATTERNcan match the test backdoor's ownbash -cwrapper (the pattern is in its cmdline) — usepgrep -xor a[t]uigreet-style bracket pattern. - Don't default a "timer/session" feature to V3 — most of it is VM-testable.
A scheduled/session behaviour usually decomposes into a generic step
already covered elsewhere (e.g.
home-manager switch, exercised by every theme apply) and a specific decision (which theme, when). Stub the generic step (NOMARCHY_REBUILD=<marker>for theme-sync) and simulate time by moving the VM clock (date -s,timedatectl set-ntp false,time.timeZone = "UTC"), then assert the decision + state change headlessly.checks.auto-themedoes exactly this for #79's sunset/sunrise. Only the literal timer-fires-on-OnCalendaris truly on-hardware, andsystemd-analyze calendarvalidates that schedule. (I first mis-framed #79 as V3 — it's V2.) writeShellScriptBinscripts runset -euo pipefail(nomarchy-doctor, the menu, lifecycle CLIs). So a no-matchgrepinside$(…)(grep exits 1 → command-sub fails → abort) and a standalonecond && action(false cond → abort) both kill the script mid-run — the tell is output that stops before the final/verdict line with no error. Guard:… | grep … || trueinside$(),cmd 2>/dev/null || echo 0for captures, andifinstead of&& action. (#77 doctor hibernate section; caught by the checks.doctor VM test on first run.)- A checks.* fixture CANNOT be a writeText/toFile state file read at eval time ("path … is not valid" — flake check's eval store won't realise it): extract the logic into a pure importable file and unit-test THAT (monitor-rules.nix / checks.display-profiles is the pattern).
- Hibernation reference (Latitude / Newton, BACKLOG #76): LUKS whole
root BTRFS;
@swap→/swap; file/swap/swapfile;boot.resumeDevice= LUKS root UUID;resume_offsetfrombtrfs inspect-internal map-swapfile -r. Swap is encrypted with root (not a cleartext partition). No zram on that box yet — zram is additive for live pressure only. Installer already creates this when swapSize>0. - CI (
.gitea/workflows/check.yml) is eval-tier only (standing decision 2026-07-10): act_runner docker-compose on the Gitea VPS; no KVM there. Full VM suite is BACKLOG FUTURE #20, not NEXT — needs a separate nix+/dev/kvm runner. Container gotchas are in the workflow header (single-user Nix + nixbld users,sandbox=falsefor Stylix IFD, Nix pinned 2.31.5 vs lazy-trees, no JS actions past node20). - The Gitea instance is 1.25.4 —
on: scheduleworkflows are supported; bump.yml assumes the Actions token can push tomain(standard Gitea behaviour, but unconfirmed until the first run lands). - The git server is Gitea (gitea/act_runner via docker-compose), NOT
Forgejo — workflows are read from
.gitea/workflows/(or.github/), never.forgejo/workflows/(a whole push cycle was lost to that). - Reusable headless VM harness:
checks.*via runNixOSTest — existing examples to crib from:distro-id(boots +switch-to-configuration dry-activate),hardware-toggles(kernel cmdline/PAM assertions),battery-charge-limit(fake Mains adapter viatest_power, real udev event burst while the oneshot is active; clean inactive result plus anInvocationIDchange proves coalesced AC re-apply). AC udev hooks for a settling oneshot must usestart, neverrestart: USB-C docks emit event bursts and restarts SIGTERM the in-flight pass intostart-limit-hit. - Themed-desktop screenshots work headlessly: software-GL Hyprland
(
LIBGL_ALWAYS_SOFTWAREon virtio-gpu) +machine.screenshot()QMP dump — prototyped 2026-06-19, kept as the fallback for theme previews (§ Visual theme picker). - Hyprland needs guest GL (
virtio-vga-gl,gl=on) in interactive QEMU or the session won't start; black screen ≈ missing GL (docs/TESTING.md § gotchas). Kitty is the sole terminal (no Ghostty). - No KVM = slow, not broken; don't read slowness as failure.
Known-broken / watchlist
- btrfs-assistant "segfault" was unprivileged-only (re-diagnosed
2026-07-04): libbtrfsutil's unprivileged subvolume iteration crashes on
btrfs-progs 6.17.1 (upstream-fixed after); as root it works, and the
pkexec launcher runs it as root. The real distro bug was no polkit
agent in the session (every pkexec failed silently) — hyprpolkitagent
now ships (hyprland.nix exec-once).
checks.snapshot-guiguards the root path. Lesson: before "app X is broken", check WHO it runs as — and whether polkit prompts can render at all (§ Snapshot browse/restore). - NixOS release bump is a trap: the discarded attempt
(branch deleted 2026-06-22) hit a Hyprland OOM blocker; a redo is a
deliberate
v2, never part of routine lock bumps. state.jsonis git-tracked inside an 86 MB flake tree, so every state write re-copies the source before eval — the wallpapers-artifact split (BACKLOG LATER) is the decided fix (§ Faster switches).- Friendly theme-state load (
modules/state-read.nix, #66):builtins.tryEvaldoes not catchreadFile/fromJSONfailures — gate withpathExists+ empty/non-object checks beforefromJSON. Subtle JSON syntax errors still surface from nlohmann (line/col); field schema stays intheme.nix. mkFlake mustbuiltins.seqthe check onto the whole return set or lazy attr access skips it.
Design invariants
- Dock transitions are ordered safety operations (#100): dock in one
hyprctl --batch(external on → every internal workspace moved → focus external → internal off); undock enables internal before moving anything. The Hyprland watcher, not a shell-pipeline subshell, owns a low-levelhandle-lid-switchinhibitor until the lid is physically open; startup cleans a validated stale process group. HDMI availability may appear only aschange:sink, so freshmonitoraddedis the intent boundary for the settled PipeWire/WirePlumber reprobe + sink pick; generic audio changes must not override a manual in-dock speaker choice. - Waybar status is never color-only (item 28 sweep, iteration #69):
every status module must distinguish its states by SHAPE (glyph) or
presence (self-hide), never color alone — good/warn/bad collapse under
color-blindness. When adding a state, give it a distinct glyph or gate
the module on it; a new
classthat only recolors an existing glyph is a regression. Suppressed notification states (DND and app-inhibited) all use the bell-off glyph + @muted. - Identity themes are not traffic lights (#69): white, vantablack,
lumon, hackerman, matte-black, miasma — monochrome / mono-hue / earthy
status by design.
audit-theme-design.pytags their hue/CVD/ANSI-family findings[identity]; do not "fix" them into R/Y/G. - Import hierarchy ≠ ANSI (#70):
import-palettes.pymust not set surface==overlay when color0==color8; light color0 is often ANSI black (not a chip). Roles are first-class — never bulk-reimport shipped JSON without a hierarchy pass.
Gotchas (cost a debugging session once)
- #127 forensics/method lessons (three wrong diagnoses' worth): drive the
real code path or measure nothing (a hand-run
dpms offhas no wake path watching it, so "input didn't wake it" was guaranteed); do not read output lists as lid positions —/proc/acpi/button/lid/*/stateis the evidence (a reload can light a panel inside a shut lid, #148); and anchor forensics on the right boot — an unclean boot end (journald corruption + dirty bit on the next boot) is itself the incident marker, and--list-bootscomes before any grep. - OVMF exposes /dev/fd0 as TYPE=disk (#112): disk picker must exclude
/dev/fd*and tiny sizes; sort largest-first or blind Enter picks floppy. - Live ISO offline theme switch (#113): only the default/pinned HM generation is offline-safe; other presets may try to build the world — document the contract and fail with a network-oriented message.
- Installer HM pre-activate needs XDG_RUNTIME_DIR (+ session bus) (#123):
runuser … activateinsidenixos-enterhas no user session. Withoutmkdir -p /run/user/$UIDowned by the install user andXDG_RUNTIME_DIR(preferdbus-run-session -- activate), dconf dies with Permission denied and the bake aborts mid-way — first boot looks unthemed / missing nm-applet even though the system installed. - ISO filename is
image.baseName(#125): volumeID alone does not renameresult/iso/…; forceimage.baseNametonomarchy-…or the artifact staysnixos-live-….iso. - Install flake ref must match the ISO branch (#124): while
v1lagsmain, seed?ref=main(and compose-lock original) from main-built ISOs;NOMARCHY_FLAKE_URLmust actually rewriteinputs.nomarchy.url(was set but unused until #124). - Gum
filterreturns unmatched typed text by default; catalog-only pickers require--strictplus an independent exact-membership validation boundary. - Waybar
layer: toprenders above even real-fullscreen windows — the bar draws over a fullscreen video.layer: bottomlets the fullscreen surface cover it while the exclusive zone still reserves the bar's space in normal tiling (trade-off: floating windows can now overlap the bar strip). Set in both waybar.nix and every whole-swap jsonc (item 30). - Hyprland binds match the exact modmask: a shifted keysym (
question) needs SHIFT inmodsor the bind never fires — the keypress falls through to the focused window (§ item 26; caught on hardware, invisible to eval-tier tests). - Never kill a Wayland session-lock client (hyprlock): its crash failsafe drops to a tty instead of unlocking (§ Hibernate double-unlock).
- rofi
element-icon sizeis one value = a square cell;WxHsilently collapses and non-square icons letterbox — pre-crop images square at build (§ Visual theme picker). - WirePlumber 0.5 monitor rules can only early-match
device.api;device.product.nameetc. bind after the rule runs — surgical libcamera scoping is impossible (§ Webcam). hyprctl switchxkblayoutis a global layout flip; per-device isolation needsdevice[<name>]:kb_layoutkeywords (§ Keyboard layouts).- Waybar's clock captures the timezone at construction — a zone change
needs SIGUSR2 (watcher in
timezone.nix) (§ Automatic timezone). - Waybar
persistent_workspaces(underscore) is dead syntax silently ignored; the hyphen form is honoured and renders phantom workspaces (§ Waybar shows non-existent workspaces). - GTK4/libadwaita/Qt6 read light/dark from the portal's
org.freedesktop.appearance color-scheme(dconf), not Stylix polarity (§ GTK/Qt ignore the theme's mode). - Update order matters downstream:
nomarchy-pull(lock) thennomarchy-rebuildthennomarchy-home, or desktop changes are silently skipped against the old lock (README § 3). - Hyprland 0.55 renames
stayfocused→stay_focused(and similar underscore effects);stayfocused 1isinvalid field typeat parse (§ polkit workspace rules, 2026-07-10 hardware). - Never gate a safety listener behind the optional feature it also serves:
the display menu existed with no profiles while its blackout rescue did
not. Rofi defaults to mouse-pointer output (
monitor=-5), which is stale in clamshell mode; use focused output (-1) for keyboard-launched UI. - Hyprland 0.53 rewrote window rules:
windowrulev2is a hard error and the old rule-firstfloat, class:^…$no longer parses — both surface a red config-error banner on the default desktop. Hyprlang legacy form is now<effect> <value>, match:<prop> ^…$(e.g.float 1, match:class ^…$); effects carry a value, matchers takematch:(§ windowrule migration). - grub
loadfonts every.pf2in a theme dir — reuse a bundled DejaVu rather than shipping fonts (§ Distro branding). - Agent instructions live vendor-neutrally in
agent/(VERIFICATION, DELEGATION, THEME-DESIGN; entry AGENTS.md) —.claude/is a thin adapter (permissions + subagent defs only; skills were removed 2026-07-11). Nevergit add -Ablindly: checkgit status --shortfor genuine strangers first (settings.local.json, harness-dropped files) and commit with explicit pathspecs (§ loop hygiene).