OpenAI co-founder team member and Tesla’s former AI chief Andrej Karpathy has posted on X about the “LLM Knowledge Bases” workflow—explaining how, recently, he shifted massive token usage from “controlling code” to “controlling knowledge”: using LLMs to turn scattered papers, articles, folders, and images into an automatically maintained personal wiki. The entire process has been accumulated on his own research project to about ~100 articles, ~400k words, and has been written and updated end-to-end by an LLM. This article organizes Karpathy’s complete setup and provides a practical checklist for developers who want to replicate it.
Core idea: raw data → LLM compilation → wiki → Q&A
Karpathy’s design philosophy can be condensed into one sentence: “raw data comes in, the LLM compiles it into a wiki, the wiki then serves for the LLM to query, and the query results continue to be written back into the wiki.” The key to the whole system is shifting the human role from “taking notes” to “monitoring the notes written by the LLM.” A knowledge base is no longer manually maintained Notion or Roam Research; instead, it’s a collection of markdown files automatically written into and maintained by the LLM.
He describes how he rarely edits the wiki directly—writing in, adding links, extracting structure, checking consistency, all done by the LLM. This “LLM-driven content, human monitoring” pattern is completely different from most people’s habit of manually writing in Obsidian/Notion, and it’s the core shift of this workflow.
Step 1：Data Ingest—dump all raw data into a raw/ folder
Karpathy’s entry is simple: create a raw/ folder and dump all source materials into it—paper PDFs, news articles, code repos, datasets, images, and lecture notes. The LLM uses this folder as input and gradually “compiles” a wiki.
He specifically mentions two tools:
Obsidian Web Clipper extension—turn web articles directly into .md files and store them in raw/
Custom hotkey—download relevant images from the web to local, so the LLM can read them directly when citing them later
Key design: all external data exists in an “offline, local” form, ensuring the LLM won’t get stuck with “can’t find the original link” when querying afterward.
Step 2：LLM compiles the wiki—automatically generates categories, articles, and backlinks
Once raw/ is ready, Karpathy uses the LLM incrementally “compile” a wiki—meaning a directory structure made up of a bunch of .md files. The LLM does four things:
Write summaries of all materials in raw/
Classify the materials into concepts
Write an article for each concept
Create backlinks between articles
This process is “incremental”—new raw/ data is added, and the LLM only updates the affected parts of the wiki, without having to recompile everything. For long-term accumulating research topics (Karpathy’s own research wiki is already ~100 articles, ~400k words), incremental updates are far more practical than a one-time full compilation.
Step 3：Use Obsidian as the IDE “frontend”, and extend with plugins like Marp
Karpathy uses Obsidian as the system’s visual frontend—so you can simultaneously view raw/ materials, the compiled wiki, and derived visualizations (slides, charts). The advantage of Obsidian is that it’s already a markdown editor, naturally compatible with the .md files produced by the LLM, and supports plugin extensions.
He specifically mentions the Marp plugin—it can render markdown directly into slide formats, allowing the LLM to output not only text but also presentations.
Step 4：Q&A—make the entire wiki the LLM’s query target
After Karpathy’s wiki reaches the “~100 articles, ~400k words” scale, the most interesting capability emerges: you can ask an LLM agent any complex question, and it will go research the answer on its own, citing relevant passages from the wiki.
Originally, he expected that running at this scale would require “fancy RAG” (vector search, embedding models, re-ranking, etc.), but in practice he found: the LLM itself maintains index files and short summaries for each article; during queries, it can find relevant passages by using these indexes and summaries. At the “~400k words” scale, even without complex RAG, it still works pretty well.
This observation aligns with the industry consensus since 2024 that “vector DBs are overheating and many scenarios don’t need them”: when your knowledge base is under a million words, structured markdown plus LLM-managed indexes is already enough.
Step 5：Output—not just plain text, but markdown/slides/charts
Karpathy’s other design: he doesn’t want the LLM to only return terminal text; instead, he lets the LLM produce structured outputs—markdown files, Marp presentations, matplotlib charts, and visualized data. These outputs can be viewed inside Obsidian.
More importantly is the loop: the produced results are often “archived” back into the wiki by Karpathy, strengthening future queries. He describes it as “my exploration and queries always add up into the knowledge base”—a stateful, growing pattern, which is the opposite of the “every ChatGPT conversation starts from scratch” model.
Step 6：Linting—LLM self-checks, finds consistency issues, and new article candidates
Karpathy runs LLM “health checks” on the wiki, handling three types of problems:
Find data inconsistencies (conflicts in descriptions of the same concept across different articles)
Use web search to fill missing information
Find interesting cross-concept links and recommend new candidate articles
This linting pass is key to making the wiki “cleaner” over time—without it, automatically compiled wiki tends to accumulate contradictions and noise. The LLM performs well at this task, and it’s one of Karpathy’s reasons for believing the workflow can run long-term.
Step 7：Build your own extra tools—such as a locally hosted wiki search engine
Karpathy mentions he “vibe coded” a small search engine running on his own wiki. The tool has two use cases: (1) he directly uses the web UI to search himself; (2) more commonly, he wraps this search engine behind a CLI interface and hands it to the LLM as a tool, so the LLM can precisely hit relevant passages during large queries.
This pattern (humans set up a CLI; the LLM treats it as a tool) is a core design under agent frameworks like Claude Code and OpenAI Codex: the LLM doesn’t read everything directly, but obtains the needed subset via tools (CLI, search engine, file system).
Step 8：Future directions—synthetic data generation and model fine-tuning
When the wiki grows large enough, Karpathy proposes two advanced directions:
Use the wiki to generate synthetic data—have the LLM automatically produce Q&A pairs, tutorials, and examples for certain topics
Fine-tune a dedicated LLM using synthetic data—so your personal LLM “knows” these materials in its weights, instead of only reading them within the context window
This direction pushes the knowledge base from “external memory” toward “internalized memory”—a next step for personalized AI. But Karpathy himself also admits this requires more infrastructure and is still in the exploration stage for now.
Karpathy’s “Idea File” concept: share ideas, not code
After the post went viral, Karpathy introduced a new concept in later posts: “idea file.” In the LLM agent era, instead of sharing specific code, share “ideas,” so the other party’s agent can be customized and built for you.
He put the “idea file” for these LLM Knowledge Bases into a GitHub gist, intentionally keeping it abstract and leaving room for each person’s agent to play. This could be a new sharing mode for the dev community in the future—not a GitHub repo, not an npm package, but an “instruction file,” an open spec for LLMs.
Implementation advice: how readers in Taiwan can start
For Taiwan developers who want to replicate this system, a practical onboarding path:
Obsidian is free software, available for macOS/Windows/Linux, and can be downloaded from the official website
Install the Web Clipper extension in Chrome/Firefox/Edge
Choose an LLM side option: Claude Code (CLI), ChatGPT (API), or local Ollama (if you have a strong GPU)
For the two folders raw/ and wiki/, place them alongside your Obsidian vault, and add version control beyond .gitignore (in case the LLM writes something wrong and you need to roll back)
Start from a research topic you’re most familiar with—for example, “2026 crypto exchange regulatory updates” or “LLM inference architectures”; after accumulating ~30–50 articles, your Q&A capability will noticeably improve
At the end of the post, Karpathy said: “There’s room to build a great new product here—not in the form of rough script mashups like this.” For builders, this thread is both a workflow explanation and a startup angle—LLM auto-wikis, a market without a clear product winner yet.
This article—Karpathy reveals the complete method to build a personal knowledge base with LLM—was first published on 鏈新聞 ABMedia.
