LLM wiki: how to build one and keep it accurate

An LLM wiki is a personal knowledge base that an AI agent writes for you. You collect raw sources (articles, papers, transcripts, notes) in a folder. The agent reads them and maintains a set of interlinked markdown pages on top: one page per concept, person or source, plus an index. When you add a source, the agent updates every page it affects. When you ask a question, it answers from the wiki and can file the answer back as a new page.

The idea comes from Andrej Karpathy. On April 2, 2026 he posted on X about "LLM Knowledge Bases", and two days later he published a longer version as an "idea file" on GitHub, written to be pasted straight into an agent such as Claude Code or Codex. His contrast is with RAG, where a model retrieves chunks of your files on every question and nothing builds up. In an LLM wiki the knowledge is "compiled once and then kept current", so cross-references and contradictions are already worked out before you ask.

Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.

That line from the gist sums up the setup most people copy: the agent open in a terminal on one side, a markdown editor on the other, and a folder of files between them. Nothing about the pattern needs a particular editor, though. Any folder and any agent that can read and write files will do.

What an editor doesn't tell you is which pages the agent changed, and whether anyone has checked them. Knowtarium is a desktop app we're building for the editor's side of that setup. It opens your wiki folder as a bundle of plain markdown, shows every page the agent writes as a diff for you to approve, and records who wrote and who checked each page. It's in pre-order and not released yet. The rest of this guide covers the setup in plain files, with notes along the way on how Knowtarium handles it.

Karpathy describes the LLM wiki pattern in three layers:

• Raw sources. Your curated documents. They're immutable: the agent reads them and never changes them. This is your source of truth.
• The wiki. A directory of markdown pages the agent writes: summaries, entity pages, concept pages, comparisons, an overview. "You read it; the LLM writes it."
• The schema. A file such as CLAUDE.md for Claude Code or AGENTS.md for Codex that tells the agent how the wiki is organised and which workflows to follow. You and the agent refine it over time.

And three things you ask the agent to do:

Two files keep it navigable. index.md lists every page with a link and a one-line summary, and the agent reads it first on every query. Karpathy notes this works "surprisingly well" at moderate scale (around 100 sources and hundreds of pages) without any embedding search. Past that, the gist suggests giving the agent a local search tool over the markdown files. log.md is a timeline of ingests, queries and lint passes.

The gist is deliberately abstract and leaves the details to you and your agent. Here is a concrete version that works with Claude Code, Codex or Cursor. It takes about half an hour, most of it spent on the schema.

1. Create the folders

Keep sources and the wiki apart, so it's always clear which text came from the world and which from the agent.

customer-wiki/
  CLAUDE.md            # the schema (or AGENTS.md for Codex)
  raw/                 # your sources: the agent reads, never edits
    2026-09-12-interview-northwind.md
    2026-09-14-support-tickets.md
    assets/            # images downloaded next to clipped articles
  wiki/                # the agent's pages
    index.md           # one line per page, read first on every query
    log.md             # what happened and when
    overview.md
    concepts/
    entities/
    sources/           # one summary page per raw source
    answers/           # good answers, filed back in

2. Put it under git

Run git init before the first ingest. Karpathy points out the wiki "is just a git repo of markdown files", and git is also the cheapest way to see what the agent changed. More on that below.

3. Write the schema file

This is the most important file in the setup. Start short and add a rule each time the agent does something you didn't want. Anthropic's guidance for CLAUDE.md applies here: keep it under about 200 lines and make each instruction concrete enough to check. If you use several agents, write it once as AGENTS.md and add a CLAUDE.md that contains @AGENTS.md, so Claude Code loads the same rules.

# Wiki schema
 
You maintain the wiki in wiki/. The sources in raw/ are read-only.
 
## Pages
- One concept, entity or source per file. Filenames in kebab-case.
- Every page starts with frontmatter: type, title, description, sources.
- Link related pages with standard markdown links.
- Every claim cites its source with a footnote keyed by source id.
- If you can't cite it, don't write it. Mark open questions as such.
 
## Ingest (when I add a file to raw/)
1. Read the source. Tell me the 3 to 5 key points before writing.
2. Write wiki/sources/<name>.md with a summary and its source id.
3. Update the concept and entity pages it touches. Flag contradictions
   with existing pages instead of silently overwriting them.
4. Update wiki/index.md and add an entry to wiki/log.md.
5. List every file you changed.
 
## Query
- Read wiki/index.md first, then open only the pages you need.
- Answer with citations. Offer to file good answers in wiki/answers/.
 
## Lint (when I ask)
- Report contradictions, stale claims, orphan pages, missing pages
  and claims without a source. Propose fixes; don't apply them yet.

Two rules in there matter more than they look: "every claim cites its source" and "list every file you changed". They cost nothing now and they make the review step possible later.

4. Add one source and ingest it

Save an article as markdown in raw/ and ask the agent to ingest it. Karpathy prefers to ingest one source at a time and stay involved: read the summary, check the updates, tell the agent what to emphasise. Do that for the first ten sources at least. It is how you find out what your schema is missing.

5. Ask questions and keep the good answers

Ask across sources, not just about one. When an answer is worth keeping, have the agent file it in wiki/answers/ and link it from the pages it draws on.

# ingest one source
> Ingest raw/2026-09-14-support-tickets.md. Show me the key points first.
 
# ask, then keep the answer
> Using the wiki only, where do new customers get stuck? Cite pages.
> File that as wiki/answers/onboarding-blockers.md and link it.
 
# health check
> Lint the wiki. Don't change anything yet, give me a list.

6. Lint on a schedule

Once a week, or after every ten ingests, ask for a lint pass. Ask for a report first and approve fixes separately, so a health check can't quietly rewrite half the wiki. A consistent prefix in the log makes it easy to see what happened recently:

## [2026-09-14] ingest | Support tickets, August
## [2026-09-15] query | Where do new customers get stuck?
## [2026-09-20] lint | 2 contradictions, 4 orphan pages
 
$ grep "^## \[" wiki/log.md | tail -5

The LLM wiki pattern makes maintenance nearly free. That's the point, and it's also the risk. The agent can touch 15 pages in one pass, and in most setups nobody reads those 15 changes. Karpathy himself stays involved when he ingests, and for team wikis he mentions "possibly with humans in the loop reviewing updates". Most tutorials skip that part. Here is what piles up when you skip it:

• Drift from the source. A summary of a summary loses a qualifier. "In some benchmarks" becomes "consistently", and three pages later it's a fact.
• Unsourced synthesis. The agent connects two ideas in a sentence that reads well and cites nothing. You can't tell it apart from a sourced claim.
• Silent resolution of contradictions. A new source disagrees with an old page, and the agent picks a winner instead of flagging it.
• Stale pages that look fresh. A page written in May reads the same in December. Nothing says it's due for a check.
• Your own edits go out of sync. You fix a number on one page. The five pages that repeat it still have the old one.
• The linter grades its own homework. Lint is the same kind of model reading the same pages. It catches contradictions between pages, not errors that every page agrees on.

None of this makes the pattern a bad idea. It means an LLM wiki needs the same thing any codebase written by one author needs: a second pair of eyes, and a record of who wrote what.

Four habits close most of the gap. You can adopt them one at a time, in plain files, with no new tools.

1. Record provenance in the frontmatter

Every page should say who wrote it, when, and from which sources, and every claim should point at one of those sources. Footnotes keyed by a source id ([^northwind-call]) survive the agent reordering the list, which numbered footnotes don't. Here's a page using the field names of the Open Knowledge Format, so other tools can read them too:

---
type: Concept
title: Import step drop-off
description: Why new customers stall when importing their data.
generated: { by: claude-code/2.1, at: 2026-09-20T09:12:00Z }
verified:
  - { by: human:sam, at: 2026-09-20T18:40:00Z }
stale_after: 2027-03-20T00:00:00Z
sources:
  - id: northwind-call
    resource: /raw/2026-09-12-interview-northwind.md
    title: Customer interview, Northwind Traders
---
 
Northwind stalled for a week on the CSV import.[^northwind-call]

generated is who wrote the current text. verified is who checked it, and the human: prefix marks a person. Add a line to your schema: "sign every page you change in generated, never add yourself to verified". Our guide to making agents cite their sources goes further.

2. Review every agent change before you rely on it

Treat an ingest like a pull request. Commit before you start, let the agent work, then read the diff and accept or reject it. Git does this well:

$ git status --short          # which pages changed
$ git diff --stat             # how much
$ git diff wiki/concepts/     # read the actual edits
$ git add -p                  # accept hunk by hunk
$ git commit -m "ingest: support tickets"
$ git restore wiki/entities/  # throw away what you didn't accept

Don't read every page. Read the lines that changed, and check each new claim against the footnote it cites. When you approve a page, add yourself to verified with the date. The full routine is in how to review agent edits.

3. Have the agent check your edits

The review should run both ways. When you change a page by hand, ask the agent to find every page that links to it, cites the same source or repeats the old value, and to propose fixes for those pages. You then review the fixes like any other agent change. This is how your correction reaches the other five pages.

4. Give pages an expiry date

Add stale_after with an absolute date: short for prices, versions and people's roles, long for definitions. A lint pass can then list pages past their date instead of guessing. Staleness becomes a date comparison, not a judgement call.

The Open Knowledge Format is an open specification published by Google Cloud for knowledge that agents write. Version 0.2 added trust signals for exactly the case above: corpora written continuously by agents and read by other agents. An OKF bundle has the same shape as an LLM wiki (a folder of markdown files with YAML frontmatter), and it reserves the same two filenames, index.md and log.md.

Readers derive a trust tier from verified: no entries means unverified, only agents or processes means machine-confirmed, and at least one person means human-reviewed. A page never gets to declare itself trustworthy. Two small differences from the gist: the gist doesn't fix a link syntax, while OKF uses standard markdown links; and OKF's log.md lists the newest entries first, under plain YYYY-MM-DD date headings. Every field is optional, so you can start an LLM wiki in OKF and add the trust fields as you go.

Git is the baseline: it shows you what changed, if you commit before each session and read the diff after. Knowtarium is the setup we recommend once the wiki grows, because it keeps the review and the trust state next to the pages instead of in your terminal. Here is how the two compare.

If you ingest a few sources a week and read every diff, git alone can be enough. Knowtarium runs on macOS, Windows and Linux, keeps everything on your computer, and doesn't call an AI itself. If your wiki already lives in an Obsidian vault, the free migration skill turns it into an OKF bundle without touching your writing; the migration guide walks through it, and the comparison page covers how the two apps differ.