LLM wikis beat raw RAG for real knowledge work

LLM-maintained wikis beat raw RAG because they compound knowledge instead of re-deriving it.

LLM-wiki is right: the next useful knowledge system is not a bigger retrieval index, but a persistent wiki that an LLM maintains over time. Raw document retrieval answers questions, but it does not remember what it already learned, what it rejected, or which synthesis won the last round. A wiki that is continuously updated by the model turns each source, query, and correction into durable structure. That is the difference between a search tool and an accumulating system of record.

Raw RAG stops at recall

Most RAG systems are built to retrieve the right chunks at query time and stop there. That is enough for factual lookup, but it breaks down when the task is synthesis across many sources. Ask a subtle question that depends on five papers, three meeting notes, and a customer complaint thread, and the model has to reconstruct the answer from scratch every time. Nothing persists except the raw corpus.

We already know the cost of that design from everyday use. NotebookLM, ChatGPT file uploads, and similar setups can be impressive on a single prompt, then forget the hard-won structure the next time you ask a related question. The same sources get re-read, the same relationships get rediscovered, and the same dead ends get revisited. That is not a knowledge base. That is a recurring computation bill.

Compounding beats repeated reconstruction

The stronger model is a wiki that the LLM updates as it reads. Each new source should revise entity pages, strengthen topic summaries, flag contradictions, and add cross-links that make future work cheaper. Karpathy’s own framing is the key insight: the knowledge is compiled once and kept current rather than re-derived on every query. That makes the system more like software than chat.

The practical win is that answers become reusable assets. If the model produces a comparison table, a thesis, or a synthesis while answering a question, that output should be filed back into the wiki. Otherwise the best work evaporates into chat history. In a compounding wiki, every serious query improves the next one. That is exactly how you want research, competitive analysis, and long-running product discovery to behave.

Maintenance is the real bottleneck

The reason humans abandon wikis is not that they dislike writing. It is that bookkeeping eats the value. Cross-references go stale, summaries drift away from the sources, and contradictions hide in plain sight. The gist is correct to center the boring work: updating links, maintaining indexes, and logging what changed. That is what makes a knowledge base usable after month three.

The log-and-index pattern is especially persuasive. An index gives the LLM a content map; a chronological log gives it memory of recent actions and decisions. Karpathy’s suggestion that a simple, parseable log can be queried with basic tools is not a gimmick. It is a reminder that disciplined structure beats magical retrieval at moderate scale. Once you have hundreds of pages, the system needs operational memory, not just semantic search.

The counter-argument

The best objection is that a wiki maintained by an LLM can calcify errors faster than raw RAG can. If the model misreads a source, writes a bad summary, and then propagates that summary across linked pages, the mistake becomes sticky. Raw retrieval at least keeps the original documents available, while a synthesized wiki can create a false sense of certainty. For high-stakes domains, that is a real risk.

There is also a scale argument. At small scale, an index file and a few markdown pages are enough. At larger scale, you still need search, provenance, and review workflows. A wiki is not a replacement for source truth, and it is not a substitute for human judgment. It is a layer on top of the corpus, not a magical compression of reality.

That counter-argument does not defeat the approach. It defines the operating rule: the wiki must remain explicitly derived, revisioned, and auditable. The raw sources stay immutable, and the wiki stays editable. With that separation, errors are visible and reversible, while the gains from compounding structure remain intact. The risk is not a reason to keep everything in raw RAG. It is a reason to design the wiki like a maintained artifact, with provenance and review built in.

What to do with this

If you are an engineer, PM, or founder, stop treating LLMs as one-shot answer engines and start treating them as maintenance workers for a living knowledge base. Keep raw sources immutable, keep a wiki layer for synthesis, and keep a log of decisions and dead ends so the system does not repeat itself. Start small with one domain, one index, and one ingest workflow. If the model can update pages, flag contradictions, and preserve the reasoning trail, you have built something far more valuable than chat: a compounding memory for your team or your own work.