docs: #127 CLOSED — the brick was hypridle eating its own resume path
All checks were successful
Check / eval (push) Successful in 3m41s
All checks were successful
Check / eval (push) Successful in 3m41s
The incident was misdated: it did not happen before 12:51:34 Jul 15
(that boot was healthy all day) — it ended the previous boot at 10:54:22
with the power button. In that boot, running the pre-fix generation:
idle cascade on schedule (lock 10:21, dpms off 10:26, suspend correctly
skipped on AC), then hypridle logs "BUG THIS: inhibit locks < 0: -1" at
10:36:29 and its rule state is corrupted from there — the fired DPMS
listener's on-resume, then the session's ONLY dpms-on caller, never
dispatches again despite provable real input (two blind hyprlock PAM
attempts) and three output remove/re-add cycles. The wedged-compositor/
DRM-master lead is contradicted: Hyprland tracked hotplug live through
the whole window, PAM worked, kernel spotless. The shipped fix (a2151f4,
input wakes DPMS compositor-side) closes exactly this mechanism.
Filed upstream with the full trace: hyprwm/hypridle#208 (0.1.7; their
closed #104/#128 are the same class). Watch folded into #146.
Sweep: BACKLOG #127 deleted (NOW keeps only the install-findings context
stub, flagged to Bernardo); ROADMAP gains "The idle brick's CAUSE found"
and both stale "cause still open" tails patched; checked-off #127
HARDWARE-QUEUE blocks pruned, their method lessons moved to MEMORY
§ Gotchas.
Verification: V0 (docs/backlog only). The evidence is journal forensics
on the incident hardware (two Sonnet passes, verdict on Fable), quoted
in the ROADMAP entry; the fix itself was V3-proven 2026-07-16.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -31,44 +31,6 @@ post-boot sessions). Preserve separation: the installer bake failure and the
|
||||
flake pin are different root causes even when they show up on the same machine.
|
||||
(Terminal / Ghostty-on-Acer → shipped as Kitty-only, #95.)
|
||||
|
||||
### 127. `[blocked:hw]` The idle brick's CAUSE — reopened; the fix shipped, the diagnosis did not
|
||||
|
||||
**Restored 2026-07-16, same day it was deleted.** It was deleted as shipped on
|
||||
the strength of a diagnosis that was retracted hours later. The **fix** is real,
|
||||
proven and stays (`misc:{key_press,mouse_move}_enables_dpms`, a2151f4): the wake
|
||||
path had a genuine single point of failure — hypridle's `on-resume` was the only
|
||||
caller of `dpms on` in the session — and it is gone. Verified end-to-end on the
|
||||
incident hardware, including a real 10-min idle blanking and waking clamshell
|
||||
(`done enabled=1 first=DP-2`, 19:32). **None of that explains the brick.**
|
||||
|
||||
**What the incident was:** docked clamshell, idle → black, external keyboard and
|
||||
mouse woke nothing, unplugging the external did not light the panel, Caps Lock
|
||||
LED still toggled (machine alive), and **Ctrl+Alt+F3 gave no usable TTY**.
|
||||
|
||||
**What has been ruled OUT (all on hardware, 2026-07-16):**
|
||||
- *DPMS-off is not the trigger.* With hypridle driving the blank, input wakes it
|
||||
in every geometry — undocked lid-open, clamshell-docked on the sole live
|
||||
output, docked with both outputs live. The clamshell skip was reverted on this.
|
||||
- *The hypridle deadlock is not the trigger.* This was the published cause and it
|
||||
was wrong: at the incident (before 12:51:34 on Jul 15) hypridle had **no
|
||||
disconnect at all** and had been healthy since Jul 14 13:09. The "19 in 11
|
||||
days" rate that made the theory look strong was normal-teardown noise
|
||||
(see #146). It should have fired `on-resume`.
|
||||
- *The #100 undock latch* explains symptom 2 (unplugging not restoring the panel)
|
||||
and is fixed (ff5017e) — but not the blackout itself.
|
||||
|
||||
**The lead is Ctrl+Alt+F3, and it always was.** Neither DPMS-off nor a dead
|
||||
hypridle can break VT switching; a wedged compositor still holding DRM master
|
||||
can, and it would explain the black screen *and* the dead TTY in one stroke.
|
||||
Every theory so far has been about the wake path — i.e. about a *symptom of the
|
||||
screen being black* — and none of them touch the TTY. Start there instead.
|
||||
Bernardo does not recall whether the machine had been relogged-in between Jul 14
|
||||
13:09 and the incident; that would say whether hypridle was ever a suspect.
|
||||
|
||||
**Note the fix makes this much harder to reproduce** (input now wakes the screen
|
||||
compositor-side), which is a good trade but means a repro may need the options
|
||||
temporarily off. It has not recurred since.
|
||||
|
||||
## NEXT
|
||||
|
||||
### 143. Rofi keeps the selected *row number* when the search changes, so the highlight lands on an unrelated entry
|
||||
@@ -219,6 +181,17 @@ is upstream hyprwm/hypridle#171 ("Possible deadlock in pollThr when compositor
|
||||
exits?", open since 2025-09, in neither 0.1.7 nor main) and it is a true bug:
|
||||
hypridle hangs rather than exits, so `Restart=always` never fires.
|
||||
|
||||
**A second real one joined it with #127's close (2026-07-17):** an
|
||||
inhibit-lock underflow (`BUG THIS: inhibit locks < 0: -1`, hypridle's own
|
||||
words, Jul 15 10:36:29) left the fired DPMS listener's `on-resume` permanently
|
||||
dead on 0.1.7 — the incident's actual mechanism (ROADMAP § "The idle brick's
|
||||
CAUSE found"). Reported upstream with the full trace 2026-07-17:
|
||||
https://github.com/hyprwm/hypridle/issues/208
|
||||
(#74/#104/#128 are the same class, closed before 0.1.7 evidently without
|
||||
killing it). Its impact path here is closed by the compositor-side wake fix
|
||||
(a2151f4); same watch posture — take the upstream fix when a release carries
|
||||
it.
|
||||
|
||||
**But its impact path is closed, which is why this is `[watch]` and not work.**
|
||||
The zombie only *persists* if systemd never stops the unit — which needed the
|
||||
user manager to survive logout, i.e. the stray pre-migration linger removed in
|
||||
|
||||
@@ -60,57 +60,6 @@ Everything else below stays open; order is convenience, not a gate.
|
||||
restores **only** the radios that were on before. If Wi-Fi was already
|
||||
off, it must stay off after disengage. Glyph must **not** appear when
|
||||
airplane is off.
|
||||
- [x] **#127 TESTS A + B — ANSWERED 2026-07-16 (AMD dev box), both closed.**
|
||||
TEST B ran and DPMS-off is **not** the trigger: with hypridle driving the
|
||||
blank, input woke it in all three configurations including clamshell-docked
|
||||
on the sole live output. The mitigation was reverted and #135 retired
|
||||
unbuilt, which also voids TEST A (its premise — the skip — no longer
|
||||
exists). What that leaves: Hyprland's input→DPMS options ship off, so
|
||||
hypridle's `on-resume` was the only caller of `dpms on` — a real single
|
||||
point of failure, now removed. It is **not** known to be what bricked the
|
||||
machine: the "hypridle deadlock" cause published on the day was retracted
|
||||
the same day (hypridle had no disconnect at the incident). #127 is reopened
|
||||
for its cause; ROADMAP § "#127's WAKE PATH fixed" is the full record.
|
||||
**Method note worth keeping:**
|
||||
the *first* TEST B was invalid — it ran `dpms off` by hand, which no wake
|
||||
path watches, so its "input did not wake it" was guaranteed regardless of
|
||||
hardware. Drive the real code path or measure nothing.
|
||||
- [x] **#127 fix from config — ALL 4 STEPS PASSED 2026-07-16 (AMD dev box)**
|
||||
— a2151f4 applied via `nomarchy-pull` + `nomarchy-home` on the dev box.
|
||||
**Done, no need to redo:**
|
||||
1. ✓ Both options read `int: 1` after `hyprctl reload` (which discards
|
||||
runtime keywords), so our Nix — not a keyword — supplies them.
|
||||
`nomarchy-dpms-off` is gone from the closure and the live
|
||||
`hypridle.conf` reads `on-timeout=hyprctl dispatch dpms off`.
|
||||
4. ✓ The brick condition on purpose, from config: `systemctl --user stop
|
||||
hypridle` + `hyprctl dispatch dpms off` → **keypress woke it in 4s**,
|
||||
with both outputs enabled (`DP-2,eDP-1`). The clamshell variant (DP-2
|
||||
alone) woke in 6s the same day. Wake proven with one enabled output
|
||||
and with two; DPMS-on-input is compositor-level and does not care how
|
||||
many.
|
||||
**Correction (2026-07-16, Bernardo):** the commit that added this
|
||||
entry (321ccf7) claimed that 4s run was "docked, lid OPEN" and that it
|
||||
retired the old "docked lid-open is untested" caveat. **Both wrong.**
|
||||
The lid was **shut the whole time** — `eDP-1` was lit *inside a closed
|
||||
clamshell* because a `hyprctl reload` had re-enabled it (#148). "Lid
|
||||
open" was inferred from an output being enabled, which is not evidence
|
||||
of anything: `/proc/acpi/button/lid/*/state` is, and it was never
|
||||
checked. Docked-lid-open remains **untested** — moot now that the gate
|
||||
is gone, but not tested. Do not read output lists as lid positions.
|
||||
2+3. ✓ **The real idle path, no scaffolding** (Bernardo, 19:22–19:32,
|
||||
docked **lid closed**): `19:22:58 Executing pidof hyprlock || hyprlock`
|
||||
(5 min) → `19:27:58 Executing hyprctl dispatch dpms off` — the genuine
|
||||
10-min timeout firing from the shipped config, and it **blanked**, which
|
||||
is the revert working → mouse move → `19:32:13 Resumed: … Executing
|
||||
…/nomarchy-display-wake` → `19:32:14 done enabled=1 first=DP-2`.
|
||||
`enabled=1 first=DP-2` = clamshell, the incident's exact geometry. One
|
||||
healthy hypridle (PID 683145) throughout.
|
||||
Note: step 1 used `hyprctl reload` rather than a full relogin — a reload
|
||||
re-reads the same on-disk config a new session would, so the gap is
|
||||
cosmetic, but a relogin has not literally been done.
|
||||
**Entry closed. #127 itself is reopened for its CAUSE (see BACKLOG) —
|
||||
the wake path is fixed and verified; what bricked the machine is not
|
||||
known, and the dead Ctrl+Alt+F3 is the lead.**
|
||||
- [ ] **#96 battery-limit row on threshold-less firmware (Acer M5-481T)** —
|
||||
on the Acer (live or installed session with the commit carrying this
|
||||
entry): open System (`SUPER+CTRL+I`). **Pass:** a "Battery limit" row
|
||||
|
||||
@@ -19,6 +19,27 @@ Template:
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-17 — #127 CLOSED: the brick was hypridle eating its own resume path
|
||||
- **Task:** BACKLOG #127 (NOW). Two Sonnet forensics passes over the machine's
|
||||
journal; interpretation, verdict and close-out here. Bernardo ruled: close +
|
||||
file upstream.
|
||||
- **Did:** found the incident was misdated — it ended the Jul 15 **10:54:22**
|
||||
boot via power button (unclean end), not the healthy 12:51 boot every prior
|
||||
timeline used. In that boot: `dpms off` 10:26; hypridle logs `BUG THIS:
|
||||
inhibit locks < 0: -1` at 10:36:29; from then the fired DPMS listener's
|
||||
`on-resume` never dispatches again despite provable input (blind PAM
|
||||
attempts) — and it was the session's only `dpms on` caller (pre-fix gen).
|
||||
Compositor provably alive throughout → the DRM-wedge lead is dead. Filed
|
||||
upstream (hypridle 0.1.7, trace attached); watch folded into #146. BACKLOG
|
||||
entry deleted; ROADMAP gained "The idle brick's CAUSE found" and its two
|
||||
stale "cause still open" tails were patched; [x] #127 HW-queue blocks
|
||||
pruned (method lessons moved to MEMORY §Gotchas).
|
||||
- **Verified:** V0 (docs/backlog); the *evidence* is journal forensics on the
|
||||
incident hardware, quoted in ROADMAP. The fix itself was V3-proven 07-16.
|
||||
- **Pending:** upstream issue is a watch (#146). NOW is empty except the
|
||||
install-findings context stub — flagged to Bernardo.
|
||||
- **Next suggestion:** #134 (NEXT, decided shape) is the queue head.
|
||||
|
||||
## 2026-07-17 — Hardware hints: the second (and last) first-session toast
|
||||
- **Task:** PROPOSED "Post-install hardware hints" (VISION § B), promoted by
|
||||
Bernardo in-session.
|
||||
|
||||
@@ -137,6 +137,14 @@ iteration would otherwise rediscover.
|
||||
without a hierarchy pass.
|
||||
|
||||
## Gotchas (cost a debugging session once)
|
||||
- **#127 forensics/method lessons (three wrong diagnoses' worth):** drive the
|
||||
real code path or measure nothing (a hand-run `dpms off` has no wake path
|
||||
watching it, so "input didn't wake it" was guaranteed); do not read output
|
||||
lists as lid positions — `/proc/acpi/button/lid/*/state` is the evidence
|
||||
(a reload can light a panel inside a shut lid, #148); and anchor forensics
|
||||
on the right boot — an *unclean boot end* (journald corruption + dirty bit
|
||||
on the next boot) is itself the incident marker, and `--list-boots` comes
|
||||
before any grep.
|
||||
- **OVMF exposes /dev/fd0 as TYPE=disk (#112):** disk picker must exclude
|
||||
`/dev/fd*` and tiny sizes; sort largest-first or blind Enter picks floppy.
|
||||
- **Live ISO offline theme switch (#113):** only the *default/pinned* HM
|
||||
|
||||
@@ -493,6 +493,36 @@ Design/decision records and a running log of shipped work (items marked
|
||||
V2 by render, both heads + the password dialog, via the new
|
||||
`tools/plymouth-preview.sh`. V3 (a real docked boot/shutdown) is queued: the
|
||||
fake heads exist from the start, so the rig cannot resize a canvas.
|
||||
- ✓ **The idle brick's CAUSE found (#127 closed, 2026-07-17):** the incident
|
||||
was misdated — it did not happen "before 12:51:34 Jul 15" (that boot was
|
||||
healthy all day and ended in a routine clean reboot); it ended the
|
||||
*previous* boot at **10:54:22** with the power button (unclean end:
|
||||
journald corruption + FAT dirty bit on the next boot; final seconds: dock
|
||||
unplug/replug storm 10:52:54–10:53:58, lid open 10:54:00). Every earlier
|
||||
hypridle timeline was built against the wrong boot. **Proven** from that
|
||||
boot's journal (running the pre-fix generation of 07-10): normal idle
|
||||
cascade — 10:21 lock, 10:26 `dpms off`, 10:31 suspend correctly skipped
|
||||
on AC — then at **10:36:29 hypridle logs `BUG THIS: inhibit locks < 0:
|
||||
-1`** (inhibitor-count underflow, its own words), after which its rule
|
||||
state is demonstrably corrupted: the 300s lock listener re-arms,
|
||||
re-fires (10:41:29) and Resumes (10:43:49), but the fired DPMS
|
||||
listener's `on-resume` — then the session's **only** `dpms on` caller —
|
||||
never dispatches again, through provable real input (two blind hyprlock
|
||||
PAM attempts, 10:44:21 and 10:52:40) and three output remove/re-add
|
||||
cycles. **The wedged-compositor/DRM-master lead is dead:** Hyprland
|
||||
tracked hotplug live through the whole window, PAM/fprintd worked, the
|
||||
kernel is spotless (zero amdgpu/DRM/PSR errors). The dead Ctrl+Alt+F3
|
||||
stops being load-bearing — no VT-switch trace exists at all (the
|
||||
keystrokes plausibly never left the flapping dock, and nothing would
|
||||
have re-lit the panels either way); it was only ever evidence *for* the
|
||||
wedge, and the wedge is contradicted. **Inferred, not provable from
|
||||
logs:** the underflow caused the corruption — the correlation is tight
|
||||
and hypridle flagged the bug itself. Consequence: the shipped fix
|
||||
(a2151f4, `key_press/mouse_move_enables_dpms`) closes exactly this
|
||||
mechanism — the first blind keystroke would have re-lit the screen.
|
||||
Upstream: hyprwm/hypridle#208 (filed 2026-07-17 with the full trace;
|
||||
0.1.7, same class as their closed #104/#128). The watch rides in
|
||||
BACKLOG #146.
|
||||
- ✓ **Post-install hardware hints (VISION § B, 2026-07-17):** after the #81
|
||||
"you're set" card, at most one additional "Hardware tips" toast — a
|
||||
Firmware line when `fwupdmgr` is on PATH, a Fingerprint line when
|
||||
@@ -666,7 +696,9 @@ Design/decision records and a running log of shipped work (items marked
|
||||
is provably innocent). But "an ordinary blank + a recovery that could not run"
|
||||
is **not** established: the wake path had a real single point of failure, which
|
||||
is fixed, yet nothing shows it had failed *during the incident*. The brick's
|
||||
cause is still open. See "#127's WAKE PATH fixed" below.
|
||||
cause was still open at this writing — **found 2026-07-17** (and it *was*
|
||||
the wake path failing during the incident, via hypridle state corruption):
|
||||
see "The idle brick's CAUSE found" above.
|
||||
- ✓ **The #107 compat shim is pinned (#133, 2026-07-16):** `mkFlake` prefers
|
||||
`state.json` and still accepts the legacy `theme-state.json`, because a
|
||||
machine only migrates on its next menu write — but nothing evaluated a
|
||||
@@ -827,7 +859,9 @@ Design/decision records and a running log of shipped work (items marked
|
||||
and a rate that counted normal shutdown noise. Both times the error was
|
||||
inferring where checking was available and cheap. Verify the instrument before
|
||||
believing the reading, *especially* when the reading confirms the theory.
|
||||
**Still open — #127's cause, now the lead question (BACKLOG #127).** We know
|
||||
**Was still open at this writing — resolved 2026-07-17, and the lead below
|
||||
did not survive the boot -4 forensics (see "The idle brick's CAUSE found"
|
||||
above).** We know
|
||||
the wake path had a single point of failure and we removed it; we do **not**
|
||||
know what actually bricked the machine. The **dead Ctrl+Alt+F3 is the strongest
|
||||
clue and points away from everything examined so far**: neither DPMS-off nor a
|
||||
|
||||
Reference in New Issue
Block a user