The Second Training of AI: Forging Memory Graphs with Tokens

May 21, 2026 · View on GitHub

Consuming electricity to train large models yields a Neural Network Ontology; consuming tokens to train memory graphs yields a Symbolic Network Ontology. The combination of the two is Neuro-Symbolic AI.

I. Where Did the Memories of the Agent You've "Raised" Go?

Lately, people love talking about "raising" or "training" AI agents. Feeding it massive amounts of conversations daily, making it read documents, having it write code for you, asking it to make decisions on your behalf—these are all massive Token consumers.

So, after consuming all these tokens, what is the target outcome of this "raising" and "training"?

Obviously, every inference yields directly usable results: an article, a piece of code, an analysis report. These results are immediately taken by the user, and their value is undeniable. However, in this process, another type of product is almost entirely overlooked by everyone—Memory.

The cognition that an agent accumulates through repeated interactions: your technical preferences, your project context, your logical habits in decision-making, every mistake of its that you've corrected... These are not "final products", but rather intermediate products. They determine the quality of the next interaction. An agent you've "raised" for three months is more useful than a brand-new one not because the underlying model has changed, but because its memories have accumulated.

But the problem is: where do these memories settle today?

In Markdown files. The LLM has to read them entirely every time. The longer the file, the more expensive and less accurate it becomes to maintain—a self-deteriorating cycle.
In Vector Databases. "Vegetarian" and "BBQ" are two isolated vector points. The system cannot tell which entry is outdated and which is the latest decision.
In Key-Value Caches. diet = "vegetarian" is directly overwritten by diet = "omnivore". The evolutionary trajectory of "used to be a vegetarian but not anymore" vanishes completely.

In other words, the agent experience you "raised" by spending hundreds of millions of tokens is scattered across unstructured fragments. You can't migrate it to another agent, you have to start over if you switch models, and even the same agent might "forget" it after a few months.

The tokens are spent, but the memories haven't truly precipitated. This is the greatest evaporation loss in today's token economy.

II. The Two Laws of the Token Economy

There is an emerging consensus regarding the token economy: Tokens are inherently services. They cannot be pre-produced or hoarded; they are consumed as soon as they are generated, billed by volume, and ephemeral. Tokens from different models are non-interchangeable, naturally tiered, and diverse—this is the underlying reason why the AI economy is a thriving ecosystem rather than a winner-takes-all market.

This is the First Law of the Token Economy: Tokens are services—consumption is delivery, and the value lies at the output end.

But that’s only half the story.

In the process of "raising" an agent, aside from the articles, code, and reports taken directly by the user, there is a "waste heat" of tokens being squandered—namely, the cognitive fragments generated during interactions. If these fragments remain perpetually scattered, they act as waste heat, never to be reused. But if there is a proper receptacle to collect, compress, and structure these fragments, they can crystallize into something entirely new.

This is the Second Law of the Token Economy: Tokens can crystallize—given a suitable crystallization receptacle, consumed tokens can precipitate into reusable, structured assets.

Without a receptacle, tokens are like spilled water, evaporating into nothingness. With a receptacle, tokens are like molten metal injected into a mold, cooling into a reusable component.

What is that receptacle? A Knowledge Graph.

III. Knowledge Graphs: The Crystallization Receptacle for Tokens

Why can only a knowledge graph act as this receptacle?

Because the core operations of memory—compression, evolution, and consolidation—are fundamentally operations on a relational network. Vectors are points, Markdown is lines, Key-Value caches are grids; only graphs are networks. Only on a network can you perform traversals, merges, contradiction detection, and timeline tracking.

Take the human brain as a reference: it is not a "storage drive" but a "digestive organ". During the day, it encodes new experiences as short-term memory, and during sleep, it executes three distinct processes:

Compression (Deep Sleep): A user mentioned React, Next.js, and Vercel deployment experiences in different conversations. These three isolated fragments are compressed into one higher-order cognition: "This user is proficient in React full-stack development". A hundred fragments do not constitute knowledge; one distilled pattern does.
Evolution (Dreaming): It discovers that the user said "we use MySQL" three months ago, but last week said "we migrated to PostgreSQL". Instead of deleting the old knowledge, it marks it as "superseded", preserving a complete timeline of the tech stack's evolution.
Prioritization (Trance/Daydreaming): When a user explicitly corrects the agent, saying, "I don't use Java anymore, I write entirely in Rust now" → Saliency score: 90, prioritized for consolidation. Conversely, a daily greeting like "Nice weather today" → Saliency score: 5, directly archived.

This is exactly what Anda Brain does. It is not a database, but a cognitive organ: it receives raw fragments from agent interactions and digests them through a "sleep mechanism" into a structured knowledge graph—what we call the Cognitive Nexus.

During this process, "crystallization" occurs twice:

First, in the Formation phase, the LLM extracts entities, relationships, and events from raw interaction data, turning unstructured conversational fragments into nodes and edges in the graph. This is the primary crystallization from chaos to order.
Second, in the Maintenance / Sleep phase, the LLM performs cross-event pattern extraction, deduplication, and contradiction resolution on the existing graph. This is the deep crystallization from fragments to higher-order knowledge.

Both crystallizations consume tokens, but each makes the knowledge density higher and the structure more compact.

IV. Two Kinds of Training, Two Kinds of Ontologies

Let’s zoom out and look at a symmetrical structure that the industry has largely ignored.

Over the past three years, almost all attention and capital have rushed toward one thing: Training Large Models. Consuming massive amounts of electricity and compute power to train a giant neural network on internet corpora. This network encodes the probability distribution of human language in the form of parameter weights—it is a Neural Network Ontology: probabilistic, black-box, and generalized. It gives AI powerful reasoning capabilities, but this capability treats all users equally; it holds zero memory about "you".

But the other half of the AI cognition puzzle is missing.

When you use tokens to "raise" an agent, and then use Brain to digest interaction fragments into a structured knowledge graph, you are actually conducting a Second Type of Training. This training consumes tokens instead of electricity, processes private interactions instead of public corpora, and produces entities, relationships, and evolutionary timelines instead of parameter weights—it is a Symbolic Network Ontology: deterministic, white-box, and personalized.

It endows AI with four things that no neural network, no matter how powerful, can inherently provide:

Deterministic Anchoring: LLM reasoning is probabilistic and prone to hallucination. The facts and relationships in a graph are deterministic and verifiable, providing a solid factual foundation for reasoning.
Persistent Identity: Model weights are frozen after training, treating everyone the same. The graph continuously grows a unique cognitive structure for every subject—identity, preferences, experiences, and decision histories.
Auditability: You can trace exactly "why this decision was made" along the paths of the graph. A neural network's reasoning path is a black box.
Personalization Without Retraining: You cannot retrain a massive LLM for every single user, but you can easily build an exclusive knowledge graph for each user.

Dimension	Large Model Training	Memory Graph Training
Energy Consumed	Electricity (Compute)	Tokens (Inference)
Data Processed	Internet Corpora (Public)	Conversations & Events (Private)
Output Produced	Neural Network Ontology (Weights)	Symbolic Network Ontology (Knowledge Graph)
Cognitive Role	General Intelligence: Reasoning Engine	Exclusive Cognition: Identity, Memory, Facts
Characteristics	Probabilistic, Black-box, Generalized	Deterministic, White-box, Personalized

Combining the two is the long-sought academic goal of Neuro-Symbolic AI.

This is no longer a distant academic concept. It is becoming an engineering reality in Anda Brain: The LLM (Neural Layer) is responsible for understanding messy natural language and translating it into graph operations; the Knowledge Graph (Symbolic Layer) is responsible for storing deterministic factual relations, handling contradictions, and preserving evolutionary timelines.

The neural layer provides a generalized reasoning engine, while the symbolic layer provides anchored, personalized cognition. Or more intuitively: Large models give AI the ability to think, while knowledge graphs give AI the foundation of thought—the deterministic cognition of "who I am, what I have experienced, and how my world works".

V. The Token Processing Chain: From Raw Ore to Alloy

There is an accurate assessment circulating in the industry: "The true dividend of the AI era belongs to those who can process cheap tokens into high-value tokens and sell them". In the context of memory graphs, this value chain can be clearly visualized:

Raw Tokens (Conversations, Events, Documents)
    ↓  Formation — Consumes tokens, extracts entities and relations
Knowledge Fragments (Nodes and edges in the graph)
    ↓  Maintenance / Sleep — Consumes tokens, compresses, deduplicates, evolves
High-Density Knowledge Assets (Mature cognitive graphs)
    ↓  Recall + Regeneration — Consumes few tokens, yields high-quality outputs
High-Value Token Outputs (Precise answers, decision advice, automated actions)

Every layer consumes tokens, but the unit value of the tokens increases at each layer. Isn't this the classic industrial processing chain? Raw ore → Smelting → Refining → Finished product.

Anda Brain is the smelter on this token processing chain. It smelts low-density conversational fragments into high-density knowledge graphs, making every downstream recall and inference more precise, more efficient, and more token-saving.

VI. When Memories Circulate: A New Asset Class in the Token Economy

Traditional token consumption operates on zero-reuse. If you spend millions of tokens deeply discussing a project proposal with an AI all afternoon, the cognitive value generated by those tokens usually dissipates once the chat ends. But if, during the interaction, those tokens are refined by Brain into a persistent, structured knowledge graph, the situation changes entirely:

Marginal costs approach zero. This graph can be queried infinitely. Each query only consumes a small number of tokens to locate relevant subgraphs, without needing to re-read the entire history.

Transferable and reusable. A graph is structured data. It can be exported from one agent and imported into another, even across different models. A knowledge graph you trained via Claude works just fine on GPT—because the graph carries logical relationships, not model-specific Prompt tricks.

Licensable and tradable. Imagine this: A senior general practitioner spends six months accumulating a knowledge graph containing thousands of symptom-disease-medication links through daily diagnostic discussions with an AI assistant. This graph could be licensed to an AI assistant at a rural clinic, instantly granting a novice AI the diagnostic intuition of a veteran doctor with decades of experience. The same logic applies to lawyers—a firm specializing in labor law could accumulate a graph of controversy focal points, precedent citations, and strategic deductions from thousands of cases. This would be invaluable to any entry-level legal AI assistant.

This means token consumption acquires investment properties—they are not just spent and gone; they are accumulating an appreciating digital asset. Data is the raw ore, the knowledge graph is the refined alloy—and the latter is the truly irreplaceable competitive moat.

VII. Standardized Memory: A New Business Format in the Token Economy

If we accept the premise that "memory is the intermediate product of token consumption", a new business format surfaces.

We are already familiar with the primary business models of the current token economy: closed-source model direct sales, open-source model hosting, API aggregation and distribution... Essentially, they are all selling inference services.

But Anda Brain proposes a new format that is orthogonal to inference services: Training Knowledge Memory Graphs.

Inference services sell a "one-time computation"—used once and gone.
Memory graph training sells the "accumulation of structured cognition"—the more you use it, the more valuable it gets.

This is not a replacement for inference services, but an addition—extracting and precipitating a layer of structured knowledge during every inference consumption. Just as refining petroleum isn't solely for producing gasoline, but also yields lubricants, asphalt, and chemical feedstocks—the "refining" of tokens should similarly yield multiple products.

More importantly, Anda Brain ensures this memory precipitation is standardized. Traditional Markdown memories, Prompt tricks, and vector fragments are non-standardized—tightly bound to specific agents, specific models, and specific scenarios. Knowledge graphs, however, are inherently structured, queryable, and importable/exportable. This means that, for the first time, memory assets have the foundation to be circulated and priced.

Conclusion: A Complete AI Cognition Requires Two Kinds of Training

Back to the question at the beginning: Where did the memories of the agent you "raised" go?

If your answer is "scattered across Markdown files and vector databases", then your tokens are evaporating. If your answer is "precipitating in an ever-growing knowledge graph", then you are forging a new kind of digital asset.

The AI industry has poured hundreds of billions of dollars into the first kind of training—using electricity to train large models, resulting in a "brain" that can think. But the second kind of training has just begun—using tokens to train memory graphs, cultivating a "brain" that can host identity, experience, and logic.

Consume electricity to train the Neural Network Ontology—giving AI general reasoning capabilities; consume tokens to crystallize the Symbolic Network Ontology—giving AI an exclusive cognitive foundation. The former enables AI to think; the latter lets AI know who it is and for whom it is thinking. Only when the two are united is intelligence complete.

The market for this second kind of training is just beginning.

Anda Brain is an open-source AI memory engine that implements sleep and consolidation mechanisms based on knowledge graphs, ensuring that every consumed token has the opportunity to crystallize into a reusable knowledge asset.