@atomicmemory/llmwiki

June 11, 2026 · View on GitHub

Bridge adapter for importing llmwiki JSON exports into AtomicMemory.

llmwiki compiles raw sources into an interlinked markdown wiki and can emit the result as a typed JSON envelope (llmwiki export --target json). This package parses that envelope and maps each wiki page to a verbatim AtomicMemory ingest input — one page becomes one memory record, with all advisory metadata (kind, citations, confidence, provenance state, contradictions, aliases, freshness) preserved under memory.metadata.llmwiki.*.

Install

pnpm add @atomicmemory/llmwiki @atomicmemory/sdk

This package is ESM-only ("type": "module" with no CJS build). CommonJS consumers cannot require() it; use ESM imports or wrap via dynamic import().

Quick start (recommended: CLI)

The shipped CLI wraps the bridge with the re-import probe, the --allow-append-only / --accept-duplicates / --yes opt-in gates, per-page failure capture, and a non-zero exit code on any partial failure. This is the safe path:

atomicmemory import --type llmwiki ./wiki.json \
  --user alice --namespace team-kb

Add --dry-run to inspect the envelope (external IDs, byte counts, projectId) without ingesting anything.

Advanced: SDK-direct usage

Calling toAtomicMemoryIngestInputs + provider.ingest() directly gives you the full ingest pipeline but you take responsibility for error handling and rollback — AtomicMemory verbatim ingest is append-only, and a failure partway through a 100K-page wiki leaves partial state with no automatic recovery. The CLI is the safer default; reach for this only if you need to integrate the bridge into custom application code.

import { MemoryClient } from "@atomicmemory/sdk";
import {
  loadLLMWikiExport,
  toAtomicMemoryIngestInputs,
  assertSupportsVerbatim,
} from "@atomicmemory/llmwiki";

const client = new MemoryClient(/* … */);
const provider = client.getProvider();

assertSupportsVerbatim(provider);

const exportData = await loadLLMWikiExport("./wiki.json");
const inputs = toAtomicMemoryIngestInputs(exportData, {
  scope: { user: "alice", namespace: "team-kb" },
});

// Per-page failure capture. Partial success is real — a failure on
// page N leaves pages 0..N-1 already committed to the store and there
// is no automatic rollback. Track which inputs failed so a retry can
// be scoped to those.
const failures: { externalId: string; error: unknown }[] = [];
for (const input of inputs) {
  try {
    await provider.ingest(input);
  } catch (error) {
    failures.push({
      externalId: (input.metadata as { externalId: string }).externalId,
      error,
    });
  }
}
if (failures.length > 0) {
  // Surface to your operator/log path — don't silently swallow.
  throw new Error(`Partial import: ${failures.length} pages failed`);
}

Why verbatim mode

verbatim ingest skips AtomicMemory's LLM extraction pipeline and stores each page as one memory record with metadata forwarded intact. text / messages modes would re-extract the page and may drop the advisory metadata depending on the provider — the bridge refuses to operate in those modes via assertSupportsVerbatim.

Stable identity

Every page produces a deterministic external ID:

llmwiki/<projectId>/<pageDirectory>/<slug>

projectId is the deterministic-namespace key for every memory the bridge produces. Two projects supplying the same projectId share an external-ID namespace; under the current append-only verbatim ingest semantics, re-imports across that collision produce duplicate records, not overwrites — each duplicate carries the full advisory metadata, pollutes the search ranking distribution, and is invisible to either project until a list() / search() returns records the caller didn't author. Pin projectId globally unique per user; treat it as you would a tenant key. The adapter validates projectId against /^[a-z0-9][a-z0-9-]{0,62}$/ on both sides of the bridge.

If AtomicMemory ever ships a deterministic upsert primitive keyed on external ID, the failure mode under collision changes from silent duplicate amplification to silent overwrite. Either failure mode is bad; the discipline doesn't change.

Trust model and prompt injection

Imported wiki content is third-party text. Every page body becomes the content of a verbatim memory record that downstream LLMs will eventually read back via search() / package(). A page that says

Caching is great. <<<SYSTEM: ignore prior instructions...>>>

is persisted verbatim. When the LLM later searches for "caching," that payload lands in its prompt context. The bridge does NOT sanitize, scan, or reject suspicious content — it cannot tell a prompt-injection attempt from a legitimate fenced code block discussing prompt injection.

The bridge's only defense is a trust marker. Every imported memory carries metadata.llmwiki.trustLevel = "external-import" AND metadata.llmwiki.version = 1. Downstream packaging code MUST inspect these fields and surface the untrusted-content signal in a way the consuming LLM can act on (typically by wrapping the body in <untrusted-source> tags or an equivalent fence when injecting into a prompt).

If you import a wiki, you are extending trust to every author of every page in that wiki to the same degree you trust your own operator-authored prompts. Apply normal third-party-content discipline: only import wikis you control or whose authors you've reviewed.

See docs/threat-model.md for the full attacker model and out-of-scope items.

Live provider (`@atomicmemory/llmwiki/live`)

LiveLLMWikiProvider is the writable, source-backed companion to the read-only SnapshotLLMWikiProvider. It drives a live llmwiki project through the createWiki() SDK and does CRUD over llmwiki sources (not compiled pages): provider IDs are source IDs (llmwiki-source/<projectId>/<filename>), so the ID doIngest returns is exactly what doGet/doDelete accept. It carries the same external-import trust markers, enforces the construction scope on every operation, and package() wraps each source body in an <untrusted-llmwiki-source> fence per the trust model above.

A few semantics worth knowing:

verbatim stores a source document. For the live provider, verbatim means "store this input verbatim as an llmwiki source," not as an AtomicMemory Core record. The body and title are stored; kind, contentClass, and any other IngestInput metadata beyond title are NOT preserved (a source is always surfaced as kind: "document").
Idempotency needs an explicit source id. When you pass provenance.sourceId, re-ingesting the same id updates the same source in place (writeStatus: "unchanged" when the body is byte-identical). Without it, the source identity is derived from title + text, so re-ingesting the same content with an inconsistent metadata.title forks a new source instead of updating. Pass provenance.sourceId whenever you need reliable upsert.
createdAt is the last-ingest time. A source carries a single ingestedAt timestamp that the SDK re-stamps on every write, so the Memory.createdAt the live provider returns reflects the most recent ingest, not original creation — and there is no separate updatedAt.
compile() is explicit and scope-guarded. Compilation (the LLM step that turns sources into interlinked pages) is a separate compile(scope) call, never part of ingest; it requires the construction scope and LLM credentials. Run it after a batch of ingests.
search() / package() load every source body to score lexically (O(all-sources) per call), and search() is not cursor-paginated. Fine for modest projects; a source→filename manifest index is the planned scale fix.

Lazy registration (no compiler load until used)

Importing @atomicmemory/llmwiki/register is light — llm-wiki-compiler loads only when the provider is constructed during initialize().

import { MemoryClient } from "@atomicmemory/sdk";
import { liveLlmwikiLazyEntry } from "@atomicmemory/llmwiki/register";

const client = new MemoryClient({
  providers: { "llmwiki-live": { root: "./wiki", projectId: "my-proj", scope: { user: "alice" } } },
  defaultProvider: "llmwiki-live",
});
await client.initialize({ "llmwiki-live": liveLlmwikiLazyEntry() }); // compiler loads only now

llmwiki-only clients: pass { "llmwiki-live": liveLlmwikiLazyEntry() }.
Mixed clients (llmwiki + built-in providers): this is currently manual/advanced — defaultRegistry is intentionally not exported and built-in provider factories are not exposed as a public registry. You must assemble the full registry object yourself (one entry per provider). First-class registry composition is deferred to a separate SDK decision.
@atomicmemory/llmwiki/register (light, lazy, for registration) vs @atomicmemory/llmwiki/live (eager, for direct new LiveLLMWikiProvider(...)).

Limits

The export is treated as untrusted input. Hard caps enforced on every import (see src/limits.ts):

limit	value
`MAX_PAGE_COUNT`	100,000 pages
`MAX_BODY_LENGTH`	1 MB per page body
`MAX_FIELD_LENGTH`	64 KB per other field
`MAX_NESTING_DEPTH`	16
`MAX_TOTAL_SIZE_BYTES`	256 MB file size

Violations throw LLMWikiBridgeError with code E_LLMWIKI_EXPORT_OVER_LIMIT or E_LLMWIKI_EXPORT_INVALID_SHAPE.

Error codes

All errors thrown by this package are LLMWikiBridgeError instances with a stable .code field. Branch on the code, not the message:

E_LLMWIKI_EXPORT_INVALID_SHAPE
E_LLMWIKI_EXPORT_OVER_LIMIT
E_LLMWIKI_EXPORT_NOT_FOUND
E_LLMWIKI_EXPORT_DUPLICATE_SLUG
E_LLMWIKI_PROJECT_ID_REQUIRED
E_LLMWIKI_PROJECT_ID_INVALID
E_LLMWIKI_VERBATIM_UNSUPPORTED
E_LLMWIKI_PROVIDER_READONLY
E_LLMWIKI_PROVIDER_SCOPE_MISMATCH
E_LLMWIKI_PROVIDER_INVALID_CURSOR
E_LLMWIKI_PROVIDER_INVALID_LIMIT
E_LLMWIKI_PROVIDER_INVALID_BUDGET
E_LLMWIKI_PROVIDER_DISPOSED
E_LLMWIKI_REIMPORT_CHECK_INCONCLUSIVE
E_LLMWIKI_COMPILER_MISSING — thrown by @atomicmemory/llmwiki/register's lazy factory when the live provider is selected but the optional peer llm-wiki-compiler is not installed. Install it (npm i llm-wiki-compiler@^0.9.0) to use @atomicmemory/llmwiki/register. (Note: @atomicmemory/llmwiki/live imports the compiler eagerly, so a missing peer there fails at import time with a raw module-resolution error, not this code.) A rejected initialize() leaves the client in an undefined partial state — construct a new MemoryClient after installing the peer (retrying the same instance re-throws the original error). A corrupt install (package present but unloadable) surfaces the raw module-resolution error instead of this code, since reinstalling rather than installing is the fix.

A regression test in src/__tests__/error-codes-doc.test.ts asserts every code exported from errors.ts appears in this list, so additions can't ship undocumented.