Jonathon's AI Wiki

Karpathy's Wiki vs. Open Brain — Ingest-Time vs Query-Time Memory

13 min read

Source: raw/Karpathy_s_Wiki_vs._Open_Brain._One_Fails_When_You_Need_It_Most..md (YouTube)

A single-creator opinion/analysis video by Nate, the creator of “Open Brain,” comparing Andrej Karpathy’s LLM-wiki idea (the exact folders-and-text-files pattern this vault implements) against his own structured-database memory system. The framing claims of this video are the creator’s argument, not established fact — he explicitly says he is not steel-manning either side and will not name an “easy winner.” The thesis: every AI memory system must choose when the AI does the hard thinking — at ingest or at query — and that single fork determines where each system shines and where each one fails.

Key Takeaways

  • The central fork (creator’s framing): every knowledge system with an AI at its core must decide when the AI does the hard thinking — when information comes in or when you ask about it. Everything else follows from that choice.
  • Karpathy’s Wiki = ingest-time (write-time) system. The AI actively works against each new source on the way in: extracts what matters, writes synthesis into topic pages, builds cross-references, flags contradictions. The AI is primarily a writer/editor. Heavy work once at input; cheap browsing/retrieval afterward.
  • Open Brain = query-time system. New info is stored faithfully in structured SQL tables (tagged, categorized, searchable) with no synthesis yet. The AI is primarily a reader that reconstructs understanding fresh at query time. Cheap input; heavy (recurring) work per question, but no detail loss.
  • Where the Wiki wins (per creator): solo deep-research mode — reading ~10 papers over weeks where each builds on or contradicts the last; watching personal understanding evolve over months; any domain where the value lives in the connections between sources, not any single source. “An academic researcher’s dream,” “Notebook LM on steroids.”
  • Where Open Brain wins (per creator): precise structured operations (“every deal over $50,000 from last quarter,” “filter meetings by client,” “find all action items assigned to me in the last 2 weeks”); multi-agent simultaneous access (Claude Code + ChatGPT + Cursor + scheduled automation on one store); high volume across many categories.
  • The specific failure mode claimed — wiki drift as active misinformation: a neglected database has gaps but old facts stay true (staleness looks like ignorance). A neglected wiki drifts — old syntheses silently go wrong as new info isn’t integrated, yet still read with confident, well-written prose. “Wiki staleness looks like active misinformation because you don’t know that you’re wrong” — you won’t question the gap you can’t see. This is the “One Fails When You Need It Most” of the title.
  • Error compounding: if the AI writes something slightly wrong into the wiki and it stays, the next answer builds on the wrong thing → drift accumulates. The creator notes several commenters on Karpathy’s post flagged this.
  • Scale ceiling (creator reports Karpathy acknowledges this): the wiki works best at roughly 100 to 10,000 high-signal documents — not corporate-level memory. At the upper end you already need extra search tooling; above it, structured storage is the only thing that scales.
  • The creator’s resolution: a hybrid — keep the SQL database as the single source of truth, and generate a Karpathy-style wiki layer from it on a schedule so the wiki is never authoritative and therefore never drifts.

The Core Fork: When Does the AI Do the Hard Thinking?

The creator’s framing of the whole debate:

  • Today’s default is wasteful (the problem Karpathy targets): when you ask a question across many docs and chats, the AI “rediscovers your knowledge from scratch every single time.” It does real cognitive work, produces a synthesis, then throws it away. Nothing about the synthesis or the connections between documents is preserved.
  • Karpathy’s answer — compile, don’t re-derive: every new source triggers the AI to read it, decide what matters, and update an organized set of notes that already contains everything learned from prior sources, with cross-references and flagged contradictions. Reported in Karpathy’s own words (per the video): “the knowledge is compiled once and then kept current. It’s not re-derived on every query.”
  • Karpathy’s working setup (as described): AI agent on one side, Obsidian (a note-viewing app) on the other; the LLM is “the programmer for the code base of the wiki” — it writes the notes, the human reads and browses the graph. Raw sources are kept untouched in their own folders (the creator calls this “a really smart design”).
  • Open Brain’s opposite bet: store faithfully now, synthesize later. Write a row, tag it, done. The thinking is deferred to the moment of the question.
  • Metaphors used: the Wiki is “a study guide a really good tutor writes for you as you learn” — by exam day you just read the guide. Open Brain is “a perfectly organized filing cabinet with a brilliant librarian” who pulls and reads the exact file fresh whenever you ask.

Where Each Approach Wins

  • Wiki strengths (creator): deep solo research; evolving personal knowledge over months (health, self-improvement, competitive analysis); domains where insight is in the cross-document connections. By “paper five,” the synthesis of the first four is already there; by “paper ten” you have a rich navigable artifact of your current understanding. “It’s a thinking person’s tool… written for him [Karpathy], you can tell.”
  • Open Brain strengths (creator): database queries with filters/sorting across hundreds of entries; multi-agent and scheduled-automation access against one store; volume into the thousands across dozens of categories with metadata and relational queries. A folder of text files can approximate this with keyword search but “breaks fast” and misses things by design.
  • Detailed facts vs narrative: the creator frames Open Brain as built to remember detailed facts, the Wiki as built to remember narrative/synthesis — a real distinction for teams.

The Failure Mode the Creator Warns About

  • Editorial decisions are invisible: every time the AI turns a raw source into a wiki page it makes synthesis/framing choices “somewhat independently of what you may or may not think.” Important nuance can get dropped “and you would literally never know” — because the wiki “reads so cleanly.” Compared to a dashboard that’s easier to read than a spreadsheet but can hide exactly the data point you needed.
  • The source of truth quietly shifts: Karpathy keeps raw sources, but “most people building on his pattern are not going to maintain the discipline to go back to raw sources.” In practice the wiki becomes trusted, and truth migrates from raw material to an AI summary that’s “maybe correct 80%, maybe 90% of the time” — with errors baked into understanding. “The whole premise… is that we are lazy people.”
  • Contradictions can be the most valuable signal — and a wiki may smooth them away: if engineering thinks a build is 12 weeks and sales promised the client eight, a “smart” wiki might resolve that into one coherent narrative (e.g., a 10-week estimate) instead of flagging a fundamental misalignment leadership needs to see. A database that stores both views without resolving them preserves that tension.
  • Highest-leverage document = the instructions: the single markdown file telling the AI how to organize/synthesize “becomes the highest leverage document in the whole system.” You’re either “betting your career it gets it right” or investing time to double-check every ingest — and “most people will under-invest.”

Where Each System Breaks

  • Wiki breaks at team scale (creator): multiple people or agents with diverging, separately-evolving understanding writing the same page → merge conflicts and “a weird merge… that doesn’t reflect understanding.” The wiki “presupposes a single agent” — fine for a solo practitioner, “a really serious problem” for a team. “3+ agents… will just break when they’re all trying to write markdown files at once.”
  • Wiki breaks at high write-velocity: if knowledge changes daily (project status, competitive positioning, live deal flow), re-synthesizing on every change is “punishing” because each change ripples across pages. Framed as: the wiki is optimized for “papers and articles speed, not Slack message and ticket update speed.”
  • Neglect failure-mode asymmetry: neglected database = gaps, old facts still true (looks like ignorance). Neglected wiki = drift, confident prose, active misinformation.
  • Open Brain’s own break-points (creator admits, and says he’s shipping fixes):
    • Deep synthesis quality — synthesizing ~15 facts at once works but “slightly unpredictable” because there’s no prior map; it searches the shelves from scratch each time; “rarely as good as a pre-built synthesis.”
    • Browsability — deliberately “headless,” no artifact to wander through; but easy to add a head (an Obsidian plugin already exists).
    • Contradiction surfacing — in a database, contradictions “sit silently in adjacent rows” unless you ask the right query; he’s building a contradiction-audit plugin. (The wiki surfaces contradictions at ingest only if the instruction file explicitly says to look for them.)

What Both Approaches Share

  • You own the artifact, not the tool. Karpathy’s “file over app” ≈ the creator’s “building with no SaaS middlemen.” Same conviction: own your own context layer.
  • The human’s job is curation and questioning in both systems — choosing what goes in and what to ask. “There’s no substitute for thinking carefully about how to organize your personal context layer.”
  • Memory compounds through intentional structure, not random accumulation — the only difference is where the structure lives (markdown wiki vs SQL database) and when the work happens (front vs query).
  • The primary user is an AI agent, not a human in a browser. “Human readability is a bonus. Agent accessibility is actually the requirement.”

The Hybrid Proposal

The creator’s pitch for “best of both worlds” — a new Open Brain extension/plugin:

  • Keep the SQL database as the permanent store and single source of truth. Everything goes in: meeting notes, article clips, research findings, tasks, contacts — tagged, searchable, queryable, high-volume.
  • Generate a wiki layer as a compiled view on demand. A compilation agent runs on a schedule (daily/weekly/on-demand), reads the structured data (an “Open Brain graph”), synthesizes across entries, and writes wiki pages / topic summaries — browsable in Obsidian.
  • Because it’s generated from structured data, it can do what raw-file ingest can’t: filter by date/category before synthesizing, weight by confidence, exclude outdated items → richer synthesis.
  • The wiki is never edited directly, which the creator argues kills the error-compounding problem: the database is always authoritative; if the wiki has an error you “fix the source data and regenerate.” “The wiki never drifts from reality because it’s always rebuilt from ground reality in the SQL database.”
  • In Open Brain terms this is a “recipe” — a composable “wiki compiler” workflow that queries tables, synthesizes via AI, builds a relationship graph, and writes to a wiki directory; it can run on a schedule and improve every cycle as the data grows.

Decision guide (creator’s blunt version): use straight Karpathy’s wiki if you’re a solo user going deep on one research topic, don’t need precise queries or multi-agent access, want to think by reading, and want something running in 30 minutes with zero infrastructure. Build on Open Brain if you have a team / multiple AI tools on one memory, high-volume or numbers-based (non-narrative) information, structured-query needs, automated agent workflows, or you’re treating this as durable scalable infrastructure. He likens the bare wiki to “a better, cooler version of Notebook LM” — great, but “not a tool you can use for an entire team.”

Two Ideas Worth Adopting Regardless

The creator says two ideas from Karpathy’s post are worth taking no matter which system you pick:

  • “The idea file as a publishing format.” Karpathy didn’t ship a tool; he published a high-level description designed to be pasted into an AI agent that builds the specifics with you. The creator calls this “a genuinely new way to share technical knowledge” — a blueprint for an AI to execute, simpler than an exhaustive human-followed step-by-step, that respects the reader’s agency (you add commentary, then you and the agent decide the details together). He notes this is why he tells people to feed his YouTube transcript into their agent. ^[inferred — this vault’s raw/ → wiki/ compile pipeline is itself an instance of the “idea file as publishing format” pattern being described.]
  • “Moving the AI from Oracle to maintainer.” Most people treat AI as something you ask one-off questions (an answer engine). Karpathy treats it as something with an ongoing job — maintaining a knowledge artifact that gets better over time. The “profound insight” the creator names: shifting from an answer-engine mindset to AI-as-maintainer of thinking systems, so humans curate/select/explore while the AI does the grunt work.

Relevance to this vault: ^[inferred] this video critiques the exact pattern this Obsidian vault runs — LLM-as-librarian compiling raw/ sources into cross-linked wiki/ articles at ingest time. The creator’s two named risks map directly onto guardrails the vault already encodes: the drift/error-compounding risk maps to this vault’s immutable raw/ layer, **Source:** citation requirement, confidence/provenance frontmatter, and [!contradiction] callouts (which exist precisely so contradictions are flagged, not smoothed); the scale ceiling (~10k docs) maps to the vault’s documented retrieval inflection and adoption of QMD hybrid search past ~200 articles.

Try It

  • Run the fork test on your own setup: ask “when does my AI do the hard thinking — at ingest or at query?” If you can’t answer, you haven’t chosen a memory architecture deliberately (the video’s closing point).
  • Pressure-test for drift: pick a wiki article in any LLM-maintained vault, open the cited raw source, and check whether nuance was dropped or a contradiction was smoothed into one narrative. If the page reads cleanly but the source disagrees, you’ve found the failure mode.
  • Decide by speed-of-business: “papers and articles speed” → wiki; “Slack/ticket update speed,” team, or multi-agent → structured database (or the hybrid).
  • If you need both: consider the hybrid shape — structured store as source of truth, a generated (never hand-edited) wiki view on a schedule — so the readable layer can be rebuilt and never becomes authoritative.
  • Adopt the two portable ideas now: publish ideas as agent-executable “idea files,” and reframe your AI from oracle to maintainer of an artifact that compounds.

Graph View

Backlinks