Compile Raw Notes Into a Citation-Traceable Wiki With llmwiki

Retrieval-augmented workflows re-read the same raw files for every question. llmwiki — the npm package llm-wiki-compiler, a 2k-star MIT project — moves that work to compile time: sources become durable wiki pages that accumulate citations, provenance, review state, and retrieval metadata, so agents consume a stable evidence pack instead of reconstructing knowledge from scratch.

Why This Skill Matters

The project implements the LLM Wiki pattern described by Andrej Karpathy. A two-phase LLM pipeline extracts concepts from your sources, then generates typed pages: concept, entity, comparison, and overview. Output is citation-traceable — paragraphs and claims cite source files and line ranges, and llmwiki lint validates the links.

Beyond the default concepts-and-queries layout, Configurable Lifecycle Profiles (.llmwiki/profile.json) declare entity schemas, typed relations, lifecycle states, workflows, and retrieval policy — enforced by the runtime rather than left as prompt conventions. The built-in autosci and newsroom templates show the machinery for research and editorial work. The npm package also includes an agent skill (skills/llmwiki/SKILL.md) and an MCP server, so agents can drive ingest, compile, query, and lint directly.

Installation

npm install -g llm-wiki-compiler

export ANTHROPIC_API_KEY=sk-...

The README requires Node.js 24 or newer. Anthropic is the default provider; OpenAI-compatible servers, Ollama, GitHub Copilot, Codex CLI local login, and other provider paths are documented in its configuration section.

Real Workflow: From a Folder of Notes to a Queryable Wiki

The quickstart compiles one source and opens the viewer:

llmwiki quickstart ./notes.md
llmwiki query "what are the key ideas?"
llmwiki view --open

From there the wiki grows incrementally:

llmwiki ingest https://example.com/article
llmwiki compile
llmwiki query "How does X relate to Y?" --save
llmwiki context "brief another agent on X" --json
llmwiki lint

ingest copies the source into sources/ and compile processes only what changed — unchanged sources do not flow back through the LLM. --save writes the answer into the wiki as a saved query, becoming future context. context builds a citation-aware evidence pack you can hand to another agent, and lint checks structure, citations, links, metadata, and freshness.

Tips

• Hold risky output for review: llmwiki compile --review or the review-policy config keeps generated pages as candidates until a human approves them — the README lists review-before-write as the first safety property.
• The default compile extracts 3–8 concepts per source; llmwiki compile --no-pages-limit (new in 1.5) removes the cap so a source yields the pages it supports.
• Export targets include Open Knowledge Format (a Google Cloud initiative, per the README), JSON, JSON-LD, GraphML, Marp slides, and llms.txt; imported OKF bundles are staged through the review queue by default.
• A provider without an embedding endpoint falls back to lexical retrieval, so query and context workflows keep working.
• llmwiki serve --root <project> exposes the wiki to MCP-compatible agents; read-only tools work without provider credentials.

When Not to Use This

The README draws its own boundaries: not a general static-site generator, not a heavy ontology database, and not a replacement for ad-hoc search over fast-changing raw logs. Compiling needs an LLM provider — only the read-only surfaces work without credentials. And native Windows validation is still outstanding per the README, which recommends a supported Linux environment for untrusted projects.

See the leaderboard for more skills.