Jonathon's AI Wiki

Karpathy Wiki Additions from Synthadoc (2026-05-05)

6 min read

Source: wiki synthesis: synthadoc (engine analysis), ai-research/synthadoc-axoviq-readme.md (raw repo content), vault CLAUDE.md diff (this session)

On 2026-05-05, the synthadoc engine landed as the most architecturally complete public reference for the Karpathy LLM-wiki pattern to date. After a side-by-side review against this vault’s existing schema and tooling, five specific improvements were adopted. This article documents what landed, why, and what was deliberately skipped.

Key Takeaways

  • Synthadoc’s docs/design.md formalized several patterns this vault had only sketched (status field, audit DB, query decomposition).
  • Five concrete additions landed in one session: schema field, two Python scripts, two CLAUDE.md sections.
  • Three patterns from synthadoc were deliberately not adopted: three-layer cache, job queue, ≥85% auto-resolution.
  • Net effect: the vault keeps its Claude-Code-as-runtime / Quartz-as-publish-target architecture, but inherits synthadoc’s discipline around contradiction tracking, source-path validation, and decomposition-before-retrieval.

What landed

1. status: active | contradicted | archived frontmatter field

What: A new required frontmatter field on every wiki article.

Why: The vault already had [!contradiction] callouts in article bodies, but Dataview and the audit DB couldn’t query “show me all contradicted articles” without scanning every body. Synthadoc’s frontmatter field makes the contradiction state queryable in a single hop.

Where: Updated in vault CLAUDE.md § Ingest step 7 (frontmatter schema), § Contradiction Handling (state-flip discipline), and the example articles in wiki/_examples/.

Migration: Existing 240 articles default to status: active. Articles with unresolved [!contradiction] callouts get bumped to status: contradicted during the next lint pass.

2. Query decomposition behavior

What: When a user question contains multiple distinct sub-questions (“X vs Y”, “differences between A, B, C”), split into 1-N sub-queries (cap=4) and run mcp__qmd__qmd_query against each in parallel before synthesizing.

Why: Single-query QMD against a multi-entity question collides on relevance — the article most-cited under “Cowork” wins over articles about “Computer Use” even when both are needed. Decomposition turns one weighted retrieval into N evenly-weighted retrievals.

Where: New section in vault CLAUDE.md § Query operation. Modeled on synthadoc’s search_decompose_agent.py.

Cost: N parallel qmd_query calls instead of 1. QMD is local and free; the parallelism is essentially free.

3. bin/lint-stale-sources Python script

What: Walks every wiki article, parses YAML sources:, verifies each cited path still exists on disk. Output to output/lint-stale-sources-YYYY-MM-DD.md.

Why: The vault’s existing manifest-integrity check goes raw→articles (does every source have an article?). Stale-sources goes the other direction (does every article still have its sources?). The first lint run on 2026-05-05 found 23 articles with stale sources and 35 with malformed sources: values — real provenance debt.

Where: bin/lint-stale-sources (new file). Documented in vault CLAUDE.md § Lint check 14a.

Output sample: Stale-source articles include _examples/ (synthetic refs by design — now skipped), several claude-onboarding modules referencing slugs instead of paths, a few articles citing files that were deleted during prior cleanups.

4. bin/build-audit-db SQLite audit DB

What: Python script that reads .manifest.json, wiki/log.md, wiki/questions.md, and output/health-history.json and produces a queryable SQLite database at karpathy-obsidian-vault-main-2/.audit.db. Tables: sources, produced_articles, log_entries, queries, health_snapshots.

Why: The manifest is a JSON blob — useful as a delta-tracking primitive, hard to query for cross-cutting questions (“what did we ingest from Reddit last week?”, “which articles were updated by the Sunday cloud routine?”). SQLite gives us SQL.

First-run stats: 245 sources, 215 produced articles, 343 log entries, 6 health snapshots. Source-type breakdown surfaced that 150 of 245 sources have no source_type set (mostly pre-2026-04-15 ingests, before the field was introduced).

Where: bin/build-audit-db (new file). Run after every ingest, or manually for a fresh snapshot.

5. Per-topic AGENTS.md (optional)

What: A file at wiki/{topic}/AGENTS.md containing LLM instructions specific to that topic — common transcript-normalization quirks, what NOT to ingest, source-quality bias, terminology disambiguation. Prepended to ingest decision prompts when the new source’s primary topic matches.

Why: The vault already has a global CLAUDE.md (vault schema) and per-topic _index.md (descriptive). Neither is a directive ingest aid. Synthadoc’s AGENTS.md fills that gap. Made optional in this vault — adopt only when a topic has accrued enough convention drift that re-stating rules in every ingest would be wasteful.

Where: New section in vault CLAUDE.md § Per-topic AGENTS.md. Initial AGENTS.md created for claude-ai/ (auto-caption normalization rules), weo-ai-governance/ (privacy posture), and ai-video-content/ (Mel’s feedback rules + voice profile reminders).

What was deliberately skipped

Three-layer cache (Layer 1 embedding + Layer 2 LLM response)

Synthadoc runs the LLM directly, so per-source cost optimization matters. This vault runs on a Claude Code subscription; per-source token cost is amortized across the subscription. Provider prompt caching (Layer 3) already happens automatically. Layers 1 and 2 only become relevant if we ever swap to a programmatic provider.

Background job queue

Synthadoc’s worker handles retryable, resumable, cancelable ingests across server restarts. This vault’s “engine” is the Claude Code session — when a session ends, in-flight work is in conversation context, not a database. The gsd-* skill family handles the longer-running asynchronous work patterns. Adding a parallel job queue would duplicate without enabling new workflows.

≥85% confidence auto-resolution

Synthadoc’s LintAgent auto-resolves contradictions above the threshold and flags the rest. This vault keeps human-in-the-loop on every contradiction. The CLAUDE.md instruction is explicit: “Do not silently overwrite conflicting claims. Preserve both versions in the callout until the human resolves the contradiction.” The friction is intentional — knowledge-base trust depends on the human owning every fact-state-change. Auto-resolution is a tool for high-volume ingest where individual-fact stakes are lower.

Try It

  • Run bin/lint-stale-sources to see the current 23 stale + 35 malformed findings. The first cleanup pass will be high-leverage — most are simple fixes (slug→path, comma-list→array, prose→description).
  • Run bin/build-audit-db --stats to see source-type breakdown, top operations, and cross-cutting query basis. Useful before lint runs and at end-of-session.
  • Add status: active to your next article’s frontmatter — every new article from now on should set this explicitly. Old articles inherit active by default.
  • Try a multi-entity QMD query — e.g., “What’s the difference between Cowork, Computer Use, and Claude Design?” Decomposition should pull all three articles cleanly.

Open Questions

  • Should the _examples/ directory get a status: archived to make Dataview filters cleaner? Currently they’re skipped by lint but otherwise indistinguishable.
  • Should bin/build-audit-db run as a post-ingest hook (auto-rebuild after every compile) or stay manual? Tradeoff: 1-2s wall time per run vs guaranteed freshness.
  • AGENTS.md in claude-ai/ is large because the topic spans 94 articles with many disambiguation rules. Should AGENTS.md be sub-divided (e.g., claude-ai/cowork/AGENTS.md) once a topic crosses an article-count threshold?

Graph View

Backlinks