docs(agent): vendor-neutral agent docs — AGENTS.md entry, agent/ SoT, .claude shims
All checks were successful
Check / eval (push) Successful in 4m28s
All checks were successful
Check / eval (push) Successful in 4m28s
The repo is maintained by agents from multiple vendors, so agent instructions move out of vendor-specific locations into shared, git-tracked markdown: - AGENTS.md is the new entry point for any harness; CLAUDE.md becomes a symlink to it (Claude Code keeps working unchanged). - Skill bodies relocate to agent/: VERIFICATION.md (the enforcement rules, ex .claude/skills/nomarchy), DELEGATION.md (capability tiers light/standard/frontier, scout/runner role contracts, token economy, parallel fan-out — consolidates the CLAUDE.md model table, LOOP.md's economy section, and skill §6.5 into one place; vendor model names survive only in the per-harness mapping table), THEME-DESIGN.md (ex .claude/skills/theme, which previously lacked frontmatter). - .claude/ shrinks to a thin Claude Code adapter: settings, subagent defs, and skill shims that route into agent/. - Maps updated: agent/README.md (instructions vs state vs adapters), docs/README.md, README.md layout tree. Cleanup: stale old_distro .gitignore entry dropped (dir long gone); local result*/__pycache__ artifacts removed; JOURNAL.md rotated (29 recent entries kept, 120 older moved to agent/JOURNAL-ARCHIVE.md, rotation rule documented in the header). Verification: V0 (docs/meta only) — nix flake check --no-build exit 0 with the tracked symlink; grep sweep confirms no vendor model names outside DELEGATION.md's mapping table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
178
agent/VERIFICATION.md
Normal file
178
agent/VERIFICATION.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# Verification — the enforcement rules
|
||||
|
||||
Nomarchy's promise to its user: a rock-stable, fully functional, beautiful
|
||||
workstation that is reproducible, easy to recover, and never requires the
|
||||
user to become a Nix expert. Every rule below exists to protect that
|
||||
promise. A change that works but degrades stability, aesthetics, or
|
||||
user-simplicity is a regression, not a feature.
|
||||
|
||||
This document is an **enforcer**, not the workflow itself: the ladder and
|
||||
iteration protocol live in `agent/LOOP.md`, VM instructions and gotchas in
|
||||
`docs/TESTING.md`. It applies to **every** change to this repo — features,
|
||||
fixes, theming, module changes, lock bumps, docs, backlog grooming.
|
||||
"Small" or "obvious" changes are exactly where verification gets skipped,
|
||||
so they trigger it too. If this file and those docs ever disagree, fix the
|
||||
discrepancy in the same or a follow-up commit so they can't disagree
|
||||
twice.
|
||||
|
||||
## 1. Preflight (once per session)
|
||||
|
||||
Before starting work, establish what verification tier this environment
|
||||
can reach, so you never promise verification you can't deliver:
|
||||
|
||||
1. Linux x86_64 host? `/dev/kvm` present and readable?
|
||||
2. Enough free disk for an image/ISO build (multi-GB)?
|
||||
3. Network access for a cold Nix store?
|
||||
|
||||
If the environment cannot reach V2 (no KVM, no disk, etc.): say so
|
||||
immediately, do the V0/V1 work honestly, mark the change **"V2 pending"**
|
||||
exactly as you would mark a hardware-blocked change "V3 pending" (§4),
|
||||
and stop short of claiming the change is done. Never simulate, guess, or
|
||||
describe what a VM test "would" show.
|
||||
|
||||
## 2. The verification ladder (enforcement rules)
|
||||
|
||||
Climb the V0–V3 ladder as defined in `agent/LOOP.md`. Four non-negotiable
|
||||
enforcement rules on top:
|
||||
|
||||
1. **V2 is mandatory for anything user-visible.** If a user of the
|
||||
installed system could perceive the change — behavior, layout, colors,
|
||||
keybinds, timing, error messages — it must be exercised in the local
|
||||
VM before commit. Docs-only, comment-only, or agent-notes changes may
|
||||
stop at the tier LOOP.md assigns them; user-visible changes may not.
|
||||
2. **Every "done" report names the tier reached and shows the evidence.**
|
||||
Evidence means: the command run and its relevant output, the checklist
|
||||
items exercised, and for visual work the screenshots viewed (§3). "It
|
||||
builds" is a V1 claim, not a V2 claim. Never let a report imply a
|
||||
higher tier than was actually reached.
|
||||
3. **A failed or flaky test is a result, not an obstacle.** Distinguish
|
||||
real failures from environment flakes using the known-gotchas section
|
||||
of `docs/TESTING.md` (e.g. no-KVM slowness, missing guest GL). If you
|
||||
cannot confidently classify a failure, report it as unresolved — do
|
||||
not retry until green and report only the green run.
|
||||
4. **VM runs are headless and unattended.** Use the repo's headless
|
||||
harness — `tools/test-live-iso.sh` and `tools/test-install.sh` for
|
||||
boot/install runs, `tools/vm/qmp.py` for programmatic VM control and
|
||||
`tools/vm/vncshot.py` for screen capture — never a graphical VM window
|
||||
or any flow that needs a human at the console. The human is not part
|
||||
of the test loop: do not pause mid-run to ask them to look at the VM,
|
||||
click something, or confirm what is on screen. A run must complete on
|
||||
its own and leave auditable artifacts behind (logs, serial console
|
||||
output, exit codes, screenshots), with every wait bounded by a timeout
|
||||
so a hang becomes a recorded failure instead of a stalled session.
|
||||
Prefer scripted assertions (process up, file exists, service/D-Bus
|
||||
state, the checks in `tools/`) over eyeballing; where judgment is
|
||||
genuinely needed — visual quality — *you* view the captured
|
||||
screenshots (§3), not the human. The human reviews evidence in the
|
||||
final report, never the live run.
|
||||
|
||||
### Regression scope after a change
|
||||
|
||||
Re-running the full checklist for every change wastes VM time; running
|
||||
nothing invites regressions. Default rule:
|
||||
|
||||
- Always: the session-sanity items (boot to session, bar renders).
|
||||
- Plus: every checklist item touching the layer you changed.
|
||||
- Plus: the theming end-to-end item whenever theming plumbing changed,
|
||||
even indirectly (palette generation, symlinks, reload hooks).
|
||||
- Lock bumps and toolchain changes: run the full checklist — their blast
|
||||
radius is unknowable by construction.
|
||||
|
||||
## 3. Visual verification protocol
|
||||
|
||||
Visual quality is a core feature of Nomarchy, so "it probably looks fine"
|
||||
is never verification. A visual/UI change is not V2-verified until all of
|
||||
the following are true:
|
||||
|
||||
1. **Before/after screenshots** of the changed surface were captured
|
||||
headlessly — `tools/theme-shot.nix` for reproducible theme renders,
|
||||
`tools/vm/vncshot.py` (driven via `tools/vm/qmp.py`) for captures from
|
||||
a running VM. No VM window, no human interaction. Capture the "before"
|
||||
from the base branch or prior generation, not from memory.
|
||||
2. **Scripted checks first**: run `tools/check-theme-contrast.py` and
|
||||
`tools/audit-theme-design.py` against the affected theme(s) before any
|
||||
eyeballing — machine-checkable legibility/design violations should
|
||||
never survive to the judgment stage. Use `tools/vm/gap-analysis.py`
|
||||
where it applies.
|
||||
3. **Two themes**: repeat the "after" capture under at least two themes,
|
||||
one with a generated palette and one whole-swap theme (e.g.
|
||||
summer-night). These exercise different code paths in the bar/launcher
|
||||
theming; a change that looks right under one can silently break the
|
||||
other.
|
||||
4. **You actually viewed the images** — open the screenshot files and
|
||||
look at them. State concretely what you inspected: alignment, spacing,
|
||||
contrast/legibility against the palette, icon rendering, no clipped or
|
||||
overlapping elements, and that the change looks intentional next to
|
||||
the "before".
|
||||
5. Keep the screenshots in the run's working area and reference their
|
||||
paths in the report, so the human can audit the same evidence.
|
||||
|
||||
If the VM cannot render the surface faithfully (known GL/compositor gaps
|
||||
in the guest — see `docs/TESTING.md`), that specific visual aspect is
|
||||
hardware-blocked: verify everything the VM *can* show, and queue the rest
|
||||
per §4.
|
||||
|
||||
## 4. Hardware-blocked checks
|
||||
|
||||
Some checks genuinely require real hardware (GPU behavior, multi-monitor
|
||||
hotplug, audio devices, power/suspend, firmware). For those:
|
||||
|
||||
1. Add an entry to `agent/HARDWARE-QUEUE.md` with: what changed, **exact**
|
||||
reproduction steps a human can follow verbatim, the expected
|
||||
observation (what "pass" looks like), and the commit hash once known.
|
||||
2. Mark the commit body **"V3 pending: <one-line summary>"**.
|
||||
3. Say it plainly in your report. A hardware-blocked check is not a
|
||||
failure and not something to hide — hiding it is the failure.
|
||||
4. When the human reports back, close the queue entry in the next commit
|
||||
and record the outcome; if it failed on hardware, that's a new bug at
|
||||
the top of the backlog.
|
||||
|
||||
Do not use the hardware queue as an escape hatch: if a check *can* be
|
||||
done in the VM, it must be. "The VM is slow" does not qualify.
|
||||
|
||||
## 5. Maintenance work
|
||||
|
||||
Maintenance follows the same ladder:
|
||||
|
||||
- **Flake lock bumps**: treat as maximum-blast-radius changes. Build,
|
||||
boot the VM, run the full regression checklist, and do a visual
|
||||
spot-check of the session (themes can shift with upstream package
|
||||
changes). Never merge a lock bump on "it evaluates".
|
||||
- **Theme imports / new themes**: import via `tools/import-palettes.py`,
|
||||
then the full §3 visual protocol; additionally verify the theme-switch
|
||||
round trip (into the new theme and back out).
|
||||
- **Docs drift**: run `tools/check-option-docs.py` after any change that
|
||||
adds or modifies options, and fix drift in the same commit as the code
|
||||
change that created it. Doc-only fixes are V0 — but verify any command
|
||||
you document by actually running it.
|
||||
- **Backlog grooming / agent-notes**: V0; keep entries consistent with
|
||||
the conventions in `agent/`.
|
||||
|
||||
## 6. Guarding the philosophy
|
||||
|
||||
Before committing, check the change against the distro's promises
|
||||
(`agent/GOALS.md` is the full statement):
|
||||
|
||||
- **User is not a Nix expert.** If the change requires the user to write
|
||||
or read Nix to use the feature day-to-day, redesign it. Configuration
|
||||
the user touches must stay in the simple, documented surface the repo
|
||||
defines.
|
||||
- **Rock-stable and recoverable.** Prefer boring, reproducible mechanisms
|
||||
over clever ones. Any change that could break boot or the session must
|
||||
have an obvious rollback story (NixOS generations count, but say so).
|
||||
- **Aesthetics are load-bearing.** A functionally correct but visually
|
||||
regressive change fails review by definition — that's what §3 is for.
|
||||
|
||||
When a requested change conflicts with these promises, stop and raise the
|
||||
conflict instead of implementing it quietly.
|
||||
|
||||
## 7. Reporting format
|
||||
|
||||
End every unit of work with a short report containing:
|
||||
|
||||
1. What changed (one paragraph, plain language).
|
||||
2. Verification tier reached, with evidence (commands + key output,
|
||||
checklist items run, screenshot paths viewed).
|
||||
3. Anything pending: "V2 pending" (environment) or "V3 pending"
|
||||
(hardware, with queue entry reference).
|
||||
4. Follow-ups added to the backlog, if any.
|
||||
Reference in New Issue
Block a user