Jonathon's AI Wiki

CLAUDE.md Context Rot Cleanup — the doc-cleanup Skill (aka-luan)

11 min read

Source: raw/reddit-1us9hfz.md (r/ClaudeAI, u/eliaff, score 151 / 29 comments, posted 2026-07-10) — cites github.com/aka-luan/doc-cleanup, checked directly against the live repo on 2026-07-10 (MIT, 26 stars, single SKILL.md at skills/doc-cleanup/SKILL.md, matches the post’s description). A second, independent pattern below is sourced from raw/Every_Prompt_You_Send_Drags_18_384_Words_Of_Junk._Here_s_How_I_Cut_It..md (Nate B Jones, AI News & Strategy Daily, 2026-07-16).

After three weeks building a product almost entirely with agents, a disciplined documenter asked Claude to evaluate what in the repo’s own required-reading docs was hurting its reasoning — expecting a clean bill of health. Instead Claude found the docs were 75% finished-work history masquerading as current instructions, plus several specific rot patterns that had been silently misleading every agent session. The author turned the audit into a repeatable Claude Skill rather than a one-off cleanup.

Key Takeaways

  • Over-documenting is not the same as documenting well. The author was disciplined the whole three weeks — milestones checked off, decisions logged, gotchas recorded — and still ended up with docs that actively degraded agent reasoning. Volume and currency are different axes.
  • The three required-reading docs totaled 1,085 lines; ~75% was finished work — fully-checked checklists, phase-completion logs, old verification notes. Every session paid the token cost to read history nobody needed anymore.
  • Agents trust these files more than they trust their own searches — by design. That’s what the files are for. But it means a doc doesn’t have to be wrong in an obvious way to cause damage; it only has to be slightly behind the code.
  • Five concrete rot patterns surfaced (see below) — each one a distinct failure mode, not just generic staleness.
  • The cleanup is now a repeatable skill, not a one-off: inventories the markdown files, verifies every claim against the actual code before flagging it, reports findings, and only rewrites after human approval. Nothing is deleted — history moves to docs/archive.
  • Result: 1,085 lines of required reading cut to 266 (about 75% reduction) while preserving the archived history.
  • Verified via direct repo check (2026-07-10): the public README describes the same mechanism in more formal terms — six specific rot patterns (completed-work logs, executed plans never rewritten, internal contradictions, stale facts diverging from code, dead file paths, authority drift) worked through five phases (inventory, diagnosis, reporting, execution, verification), with a mandatory approval gate before any change lands.

The Five Rot Patterns Found in the Original Post

  1. Pure finished-work history disguised as instructions. Fully-checked checklists and phase-completion logs sitting in the same file as active rules, with no separation between “what we did” and “what to do.”
  2. The same fact stated two contradictory ways in one file. The pricing doc listed enabled payment methods differently in two places — and one version referenced a line number that no longer existed.
  3. A table “corrected” by a footnote three paragraphs below it, but the table itself never edited. The author’s own framing: “Guess which one an agent reads.” Agents don’t reliably read past the first authoritative-looking statement.
  4. Stale status labels. Two features marked “awaiting merge/deploy” had actually been on main for days — the doc hadn’t been touched after the merge landed.
  5. A dead cross-repo path that silently failed every time. A hard rule in AGENTS.md pointed at a sibling repo via a Windows path (A:\Dev\...) — left over from before the author moved to WSL. No agent ever flagged the rule as broken; each one just silently failed to follow it.

How the Skill Verifies Before Flagging

The skill doesn’t just pattern-match for staleness — it checks claims against ground truth before surfacing them:

  • “Awaiting merge” claims are checked with git merge-base against the actual branch state, not taken on the doc’s word.
  • File-path references are checked with grep against the real filesystem, catching dead paths like the WSL-migration Windows path above.
  • Contradictions are surfaced for human review rather than auto-resolved — the skill reports, it doesn’t silently pick a winner.
  • Nothing is deleted. Finished-work history moves to docs/archive rather than disappearing, preserving the audit trail while clearing it out of the always-loaded read path.
  • A mandatory approval gate sits before any rewrite. The skill’s five-phase flow (inventory → diagnosis → reporting → execution → verification, per the repo’s own README) stops before the execution phase until a human signs off.

Relationship to First-Party Tooling

This is a community pattern filling a gap Anthropic is now also addressing natively. Claude Code’s /checkup command (announced by Boris Cherny, 2026-07-08) runs a similar one-pass audit — deduplicating CLAUDE.md, reorganizing a bloated root file into nested CLAUDE.md files plus skills, and cleaning up unused skills/MCP servers/plugins — with its own confirm-before-change gate. /doctor diagnostics cover an overlapping but narrower slice (proposing trims of content Claude could re-derive from the codebase). See the CLI Reference for the authoritative first-party detail on both. The doc-cleanup skill predates neither by much and is more narrowly scoped to the documentation-rot diagnosis specifically (the six named patterns above), whereas /checkup bundles doc cleanup into a broader maintenance sweep (skills, MCP servers, hooks, update, auto mode, pre-approvals) — worth running both rather than treating either as a full replacement for the other.

A Second Independent Pattern — Nate B Jones’s Harness Cleaner (2026-07-16)

A separate creator, independently, built a broader-scoped cleanup skill covering the whole harness (custom instructions, project files, saved prompts, memory, skills, tools, permissions — not just CLAUDE.md/AGENTS.md text). Same underlying diagnosis as doc-cleanup and /checkup above — accumulated rules quietly degrade newer, more capable models — but a distinct methodology and, notably, a controlled before/after result the other two patterns don’t have.

  • Audit numbers from his own setup: 66 reusable skills and 172 instruction-related files; one route alone carried 27,000 description characters against Codex’s 8,000-character discovery budget; one skill-to-skill chain totaled 18,000 words before the model even started the actual task.
  • Six principles distilled from the exercise:
    1. Map the harness before cleaning it. Give every control a row: where it lives, when it loads, what job it does, who owns it, whether there’s evidence it still helps, and what it breaks if misused.
    2. Blame the right layer. Before adding another rule to fix a bad result, ask whether the model failed or the harness (instructions/skills around it) failed — most accumulated bloat comes from blaming the model for harness-caused failures.
    3. One rule, one home, one owner. He found the same “cite sources correctly” instruction duplicated across 15 top-level skills — each copy a separate place the rule could drift out of sync with the others.
    4. Load specialist knowledge only when the work needs it, not all at the start of every session — six editorial guides loading on every writing task meant research-quality prompts were competing for attention with irrelevant YouTube-formatting guidance.
    5. Hard requirements need hard checks. Rules with a testable yes/no shape (a word-count limit, a required JSON shape) belong in an enforceable schema the system can verify, not in prose the model has to remember to honor.
    6. Build for the specific model + product combination actually doing the work — Fable 5 in Claude.ai, Fable 5 in Claude Code, and Fable 5 via the raw API don’t share a harness, and neither do GPT-5.6 in ChatGPT Work vs. Codex vs. the API; core requirements (facts needed, permission boundary, proof-of-done) stay constant, but how they’re delivered has to match the specific surface.
  • The controlled result that neither doc-cleanup nor /checkup has produced: he ran the same task through Fable 5 with two harness weights. The compact harness (goal + facts + permission boundary + finish line only) finished the delivery requirements correctly 3 times out of 3. The thick harness (the same, plus the full method, a scoring system, an eval plan, and a classification scheme) produced richer analysis but failed delivery twice — once breaking the required JSON shape, once breaking a word-count limit.
  • Cross-model failure-mode difference he observed: Fable 5’s failure mode showed up after the method got too heavy relative to the delivery job (it tried to do too much and blew the format). GPT-5.6 in Codex failed earlier — while still searching for the right method, overwhelmed by having to route through a huge harness before it had even found its footing. Same root cause (harness too heavy), different symptom per model.
  • Where to find it: Jones published the full “cleaner” skill on his Substack; not independently verified or installed at ingest time (unlike the aka-luan repo above, which was checked directly against its live GitHub source).

How this relates to the patterns above: doc-cleanup and /checkup are scoped to markdown documentation specifically; Jones’s cleaner is scoped to the full harness (skills, tools, permissions, memory included) and is the only one of the three with a controlled compact-vs-thick reliability comparison. Worth treating as complementary rather than redundant — a documentation-hygiene pass and a full-harness audit catch different classes of bloat.

Try It

  1. Install the skill from github.com/aka-luan/doc-cleanup (MIT license, single SKILL.md) and point it at your project’s required-reading docs (CLAUDE.md, AGENTS.md, onboarding docs).
  2. Before running it for real, skim the six rot patterns above against your own docs — the table-vs-footnote and dead-cross-repo-path patterns are easy to miss by eye.
  3. Let the skill report before it rewrites — review the flagged items, especially any “awaiting merge” or file-path claims, before approving.
  4. Route history to docs/archive instead of deleting it, matching the skill’s default behavior.
  5. If you’re already on a build with /checkup (2026-07-08+), compare its output against doc-cleanup’s report on the same repo — they target overlapping but not identical problems.
  6. Re-run periodically, not just once — the same rot (stale status labels, contradicted facts) will reaccumulate as a project evolves.
  7. Before adding a new rule after a bad result, ask Jones’s question first: did the model fail, or did the harness fail? If you can’t tell, that’s itself a sign that the harness isn’t mapped clearly enough to answer.
  8. If you have a rule with a testable yes/no shape (a length limit, a required output format), move it into a schema or automated check rather than another sentence of prose — per principle 5 above.

Open Questions

  • No evidence of official Anthropic affiliation. This is a community-built skill (single author, aka-luan), not a first-party tool — treat it with the same scrutiny as any other community skill before installing.
  • Nested CLAUDE.md hierarchies untested here. The source describes a flat multi-doc setup (three required-reading files); unclear from the post whether the skill has specific handling for project/user/enterprise CLAUDE.md layering.
  • Not yet run against this vault’s own wiki/CLAUDE.md or karpathy/CLAUDE.md. A natural dogfooding candidate, unexplored as of this ingest.
  • Nate B Jones’s cleaner skill is not independently verified. Unlike aka-luan/doc-cleanup (checked against its live GitHub repo), the Substack-published cleaner has not been fetched, installed, or tested at ingest time — treat the 6-principle framework and the compact-vs-thick result as reported, not verified.
  • The compact-vs-thick harness test is a single trial, not a benchmark. “3/3 vs 2 failures out of some number of thick-harness runs” is Jones’s own account of one comparison on one task on Fable 5 — informative but not a controlled study with a stated sample size.

Graph View

Backlinks