From "Compiling Knowledge" to "Forging the Brain" —— Anda Brain Responds to Karpathy's "LLM Knowledge Bases"

Andrej Karpathy: LLM Knowledge Bases

In April 2026, Andrej Karpathy shared his latest exploration results and workflow shift: his Token consumption is transitioning from "operating code" to "operating knowledge". He demonstrated a minimalist system—using LLMs to "compile" raw materials into a structured Markdown wiki, and maintaining this living knowledge base through regular "knowledge linting".

Karpathy’s insight is exceptionally sharp. He pinpointed a major misconception in the current AI industry: we are obsessed with making LLMs "generate content", while neglecting their far more powerful capability to "organize discoveries". However, as an extension of a personal research workflow, the specific solution he proposed (Markdown flat files + a single Agent + ultimate internalization into weights) exposes unignorable architectural bottlenecks when scaling toward true enterprise-grade intelligence.

This is also the logical starting point for the creation of Anda Brain. We are in complete agreement with Karpathy that "LLMs should be used to digest knowledge", but regarding "how to build the memory foundation", we offer a fundamentally different architectural answer.

1. The Consensus: LLMs are Knowledge "Compilers", Not Mere "Typewriters"

The success of Karpathy’s approach validates three core propositions of Anda Brain:

1. Knowledge must be "digested", not merely "retrieved". Traditional RAG (Retrieval-Augmented Generation) merely acts as a porter, retrieving relevant text snippets and stuffing them into the model. Conversely, both Karpathy’s wiki compilation and Brain’s Formation inherently force the LLM to perform deep structural processing at the very moment of data ingestion.
2. Structured reasoning trumps vector matching. Karpathy found that a 400,000-word wiki requires absolutely no Vector DB, because in a well-structured knowledge base, similarity ≠ relevance. Brain similarly abandons pure vector similarity matching, relying instead on the topological structure of knowledge graphs to assist LLMs in logical navigation.
3. "Knowledge Linting" is the most underestimated component. Without maintenance, knowledge rots. Karpathy has the LLM perform regular health checks on his wiki; this directly mirrors the core of Brain’s Maintenance/Sleep mechanism: automatically executing knowledge compression, deduplication, evolution, and conflict resolution in the background.

2. Divergence and Transcendence: The Limits of Flat Wikis and the Dimensional Advantage of Knowledge Graphs

Karpathy’s architecture is elegant, but at its core, it remains a single-user, single-Agent Markdown flat-file system. Building upon this premise, Anda Brain achieves a paradigm shift:

1. From "Flat Files" to "Network Graphs": Breaking the Cost Bottleneck of Knowledge Digestion

Karpathy’s wiki is a collection of Markdown files organized by directories and hyperlinks. When executing cross-document conflict detection, timeline tracking, or event merging, the Agent can only rely on repeatedly loading and reading long texts. The computational cost grows linearly or even exponentially with the volume of knowledge.

Knowledge is inherently a network. In Brain, knowledge fragments crystallize into graph nodes and edges (Entities & Relations). For example, "The user used MySQL three months ago, but migrated to PostgreSQL last week". In a flat-file system, the LLM must carefully compare textual differences; in a knowledge graph, these are simply two timestamped edge relations attached to the same entity. Brain’s sleep maintenance combines "deterministic graph algorithms + graph topology fine-tuning + LLM local higher-order reasoning", providing an overwhelming advantage in efficiency and accuracy over text-based linting.

2. From "Solo Tool" to "Organizational Intelligence": Providing a Shared Cognitive Foundation

The sharpest criticism of Karpathy’s solution from community developers is that it remains a local knowledge base trapped within a single Agent. However, as Jack Dorsey noted in From Hierarchy to Intelligence, the enterprise of the future is "a company built as an intelligence", which inevitably requires multi-Agent collaboration.

A customer pain point collected by a Sales Agent must seamlessly pass to a Product R&D Agent; a novel bug workaround discovered by a Customer Service Agent must immediately become common sense for all Support Agents. Brain’s Knowledge Graph is inherently a multi-Agent shared cognitive hub: supporting multi-Agent write access, unified sleep maintenance, and global intelligent recall. It is not just a memory file for a specific assistant; it is the "world model" for the entire company.

3. From "Internal Proprietary Format" to "Liquid Asset"

Karpathy’s wiki is deeply tethered to his personal note-taking system (Obsidian) and bespoke directory structures. This makes the knowledge non-standardized and difficult to migrate across systems.

Brain crystallizes tokens into standardized graph-structured data. This means the organizational experience and expert intuition graphs you accumulate using Claude can be seamlessly switched over to any other model with a single click. For the first time, memory acquires the attributes of a transferable, reusable, and tradable digital asset.

3. The Fundamental Disagreement: Refusing to "Hardwire" Knowledge into Model Weights

At the end of his thread, Karpathy outlined an endgame vision: as the knowledge base grows massive, it is eventually fine-tuned via synthetic data, allowing the LLM to fully "internalize" the knowledge into its model weights.

Anda Brain explicitly rejects this technical roadmap.

Internalizing knowledge into weights is akin to hardwiring RAM data directly onto a CPU processor. Not only does this violate the fundamental software engineering principle of decoupling, but it also creates a "dead end that prevents smooth upgrades". The reasons are as follows:

Dimension	Plan A: Internalize into Weights (Fine-Tuning)	Plan B: Externalize to Graph (Brain)
Model Iteration	Knowledge is bound to the old model. Switching to a next-gen model (e.g., GPT-5 to GPT-6) requires re-fine-tuning and re-injecting, with uncontrollable costs and quality.	Hot-swappable models at any time. Cognitive assets precipitate independently in the graph. Plug a new CPU into the old hard drive, and it instantly inherits all memories.
Auditability	Black-box mechanism. The model provides an answer, but you cannot trace how it reached that conclusion, nor can you easily correct it.	White-box mechanism. You can follow the logical paths of nodes and edges to pinpoint exactly "why this judgment was made", modifying flawed cognition at any time.
Evolution Tracking	New knowledge overwrites old knowledge via fine-tuning; historical evolutionary trajectories completely disappear.	Old facts are not erased but marked as "Superseded", fully preserving the developmental timeline of events.
Multi-Agent Isolation	To ensure security and privacy, every employee/team must fine-tune a dedicated LLM, leading to exploding compute and maintenance costs.	Shares the same reasoning core (LLM). By slicing query subgraphs based on permissions, costs remain extremely low.

In Brain’s architectural philosophy, the LLM is merely a pluggable "general intelligent calculator", while the knowledge graph is the true "long-term brain" that accumulates user or enterprise-proprietary cognition. Computing power should reside in the model foundation, but core cognitive assets must remain firmly in the hands of human organizations.

Conclusion: Beyond Compilation, Towards Neuro-Symbolic AI

Karpathy provided a highly insightful workflow framework: collect, compile, query, and lint. This is indeed the future of knowledge management. However, he still attempts to encompass everything using purely Connectionist approaches.

Anda Brain fills in the final missing piece for achieving enterprise-grade implementation: Neuro-Symbolic AI.

• Neural Layer (Large Language Models): Responsible for understanding chaotic natural language, generating text, and performing fuzzy intent reasoning.
• Symbolic Layer (Knowledge Graph Memory Engine): Responsible for storing deterministic factual networks, verifying causal relationships, and preserving strict developmental timelines.

The era of compilation has arrived, but it is by no means the endgame. The output of compilation should not be a pile of scattered documents, nor should it solidify into the black-box probability weights of a model; it should be a white-box, network-like castle of long-term memory that truly belongs to you.