Usage Guide

July 23, 2026 · View on GitHub

There are two recommended ways to use lilbee. Run lilbee for the TUI if you're the one driving; wire it into your agent over MCP if an AI agent is. Both cover the everyday workflow: pick models, index files, search, chat, manage the wiki. You should not normally need to touch a CLI flag, an environment variable, or config.toml by hand for either path.

Everything in Reference for advanced users further down (CLI commands, the HTTP server, env vars, the config file) is for users who need to drive lilbee from outside those two paths: CI, scripts, headless boxes, custom integrations. Skip past it unless that's you.

The TUI

lilbee (no args) launches the full Textual app: streaming chat with clickable citations, a model bar, a Task Center for background jobs, and screens for the catalog, settings, the setup wizard, and the auto-built wiki.

Press ? at any time for the keybinding cheat sheet, Ctrl+P for the Textual command palette, and /help for the slash-command catalog. Every action lilbee can take is reachable from one of those three.

First run

The welcome screen walks you through:

  1. Picking an embedding model (required for indexing your documents).
  2. Picking a chat model (optional; needed for cited answers and the conversational REPL).

Both pickers show every model the role can run: native GGUFs from the built-in catalog and, when an API key is configured, anything your provider exposes. Pulling a model installs it; selecting it assigns it to the matching role.

After that you're in chat. /add indexes documents, /crawl indexes a website (crawler extra required), /settings opens the settings tabs.

Adding documents

Press /add to add files, directories, or web pages. Paths tab-complete; the job runs in the Task Center, so you can keep chatting while indexing happens.

If a file with the same name is already indexed, add skips it. To re-index in place, remove the document first with /delete name, or pass --force from the CLI.

Search vs Chat mode

The bar above the prompt has a two-state pill that toggles between Search and Chat. F3 flips it.

  • Search (default). Every prompt goes through document retrieval first. Relevant chunks are passed to the chat model as context, and the reply ends with a Sources block of clickable citations. If retrieval finds nothing usable, lilbee says so plainly instead of answering from the model's general knowledge; switch to Chat mode when you want an off-corpus answer.
  • Chat. Retrieval is skipped entirely; the model answers directly from whatever it already knows. Useful for conversational follow-ups that don't need your library, or for talking to a model when you haven't indexed anything yet.

The toggle is disabled and forced to Chat when no embedding model is configured (Search has nothing to search against).

Model bar

The bar above the prompt shows what's active for chat and embedding, plus the mode toggle:

Chat [Qwen3 0.6B]   Embed [Nomic v1.5]   [Search | Chat]

The chat and embedding labels are searchable pickers. Click one (or focus it with Tab and press Enter) to open a modal with a search input above a virtualized list of every model that role can use. Type to filter; Enter picks the highlighted row; Escape cancels. The active model's display label sits on the button at all times so you can see what's loaded without opening the picker.

The pickers list everything the role can run: native GGUFs you already have installed plus, when the litellm extra is installed and an API key is set, whatever the SDK backend exposes for that provider. There is no separate "local-only" picker; routing happens automatically once the model is selected.

Catalog screen

/models (or /m, /catalog) opens the catalog. The top of the screen has two sub-tabs:

  • Local. Everything you can run on this machine. Native GGUF models from the developer's picks, the full HuggingFace catalog browsable by task and size, and any locally-running OpenAI-compatible backend that exposes models on its REST endpoint. Toggle between a card grid and a dense list view; both share the same search box. Pulling a model installs it; selecting an installed model assigns it to the matching role.
  • Frontier. Cloud chat models grouped by provider (Anthropic, Gemini, OpenAI, and so on). Only appears when at least one provider API key is configured, either via the Settings screen's API-Keys tab or the provider's standard environment variable. The list shows whatever the provider's SDK exposes for that key with no curation on lilbee's side. Selecting a row makes that model the active chat model and routes subsequent prompts through the cloud provider; a persistent warning appears in the model bar so it's clear when chunks are leaving the machine.

Settings screen

/settings opens a tabbed settings editor. Every value persists to config.toml (see Config file); the equivalent env vars override individual values at runtime without touching the file.

Tabs for features that aren't installed are hidden, not greyed out:

  • API-Keys appears only when the litellm extra is installed.
  • Crawling appears only when the crawler extra is installed.
  • Wiki appears only when the experimental wiki layer is enabled (cfg.wiki = true in config.toml or LILBEE_WIKI=1).

Install the relevant extra (or flip the wiki flag) and the tab shows up on the next visit.

Slash commands

All slash commands available from the TUI. Slash commands and paths tab-complete; /help opens the same catalog live.

CommandAliasesDescription
/model [name]Switch chat model. No args opens the catalog picker; with a name, switches directly or prompts to download
/models/m, /catalogBrowse the full model catalog
/add <path>Add a file or directory to the index (tab-completes paths)
/crawl [url]Crawl a URL. No args opens a dialog
/delete <name>Remove a document from the index
/remove <name>Remove an installed model
/wikiOpen the auto-generated wiki
/remember <text>Save a memory (prefix with pref: for a preference). Needs memory enabled
/memoriesBrowse, delete, or share saved memories
/setupRun the first-time setup wizard
/settingsView or change settings
/set <key> <val>Change a setting (e.g. /set temperature 0.7)
/theme <name>Switch theme
/statusShow indexed documents and config
/login <token>Log in to HuggingFace
/clearClear chat history
/cancelCancel active operations
/resetFactory reset (asks for confirmation)
/versionShow lilbee version
/help/hShow available commands
/quit/q, /exitExit

A spinner shows while waiting for the first token from the LLM.

Task Center

Background jobs (sync, crawl, wiki build, model pull) appear in the Task Center with live progress. /cancel cancels the active operation. The TUI stays responsive throughout; each inference role (chat, embed, rerank, vision) runs in its own subprocess so a stuck model doesn't lock the chat.

Wiki

lilbee analyzes the documents you've indexed and writes a wiki about them, inspired by Andrej Karpathy's LLM Wiki. Pages compound across sources instead of being one-per-document, so concepts and entities that show up repeatedly in your library get their own page with citations from every source that mentions them.

Open it with /wiki. Pages live under $LILBEE_DATA/wiki/:

DirectoryContents
concepts/One page per LLM-identified concept (e.g. braking-systems.md)
entities/One page per proper-noun entity extracted by NER (e.g. henry-ford.md)
drafts/Low-faithfulness or parse-failure pages awaiting your accept/reject
archive/Pages retired by lilbee wiki prune
synthesis/Cross-source pages produced by lilbee wiki synthesize
index.mdAuto-generated table of contents, grouped by page type
log.mdAppend-only audit trail of every build, ingest, lint, and prune

Every section is citation-verified against the source chunks and scored for embedding faithfulness; low-confidence output routes to drafts/. Plain-text concept slugs inside page bodies are rewritten to Obsidian [[wiki links]] so the graph view shows how ideas connect. The directory is Obsidian-compatible out of the box.

The wiki is built incrementally during sync (default cap of 20 changed sources per sync) so day-to-day re-ingest never churns existing concept slugs. Rebuild from scratch, lint, drafts review, and prune are also available as CLI commands (see Wiki commands) and as MCP tools.

Sessions

Conversations save automatically. The first message opens a session and names it after what you asked; each turn is appended as it lands. There is no save step.

  • The drawer (ctrl+o or /sessions) docks the session list beside the live chat. Type to filter, enter resumes, ^n new chat, ^r rename, ^d delete.
  • The Sessions tab is the same list full-screen.
  • The CLI: lilbee sessions list / show / rename / delete (see Sessions commands).

Resuming restores the transcript and switches back to the model the conversation used, if it is still installed; otherwise lilbee keeps the current model and says so.

Sessions are append-only JSONL files under <data_dir>/sessions/, one per conversation. No database; back them up or sync them like any other file. The same surface exists over HTTP (list, read, create, append, rename, delete), so a script can own a conversation the way the TUI does. The TUI, HTTP server, and CLI are one conversation space: start a chat in Obsidian, continue it in the terminal.

Agents get the same tools over MCP, but they are off by default: most agent hosts already track their own history, and the tools cost context on every request. Turn on mcp_sessions_enabled if you want an agent keeping its own saved conversations. Agent sessions stay separate from yours: they never appear in your session list, and agents cannot list or read yours. The one bridge is explicit: an agent can take over a session whose id you hand it (claim=true, which moves it to the agent's space), and POST /api/sessions/{id}/claim brings one back.

To keep nothing, turn sessions_enabled off in Settings: nothing is written to disk, ctrl+o leaves the footer, and the Sessions view says sessions are off instead of showing an empty list. That covers the TUI, HTTP server, and CLI; mcp_sessions_enabled governs the agent half independently.

When a conversation outgrows the model

By default, turns that no longer fit the model's context window stop being sent. They stay on screen; a context gauge by the prompt shows the window filling, and a rule in the transcript marks where the model's view now begins.

To keep the old context instead, turn on chat_compaction in Settings. When the window fills, lilbee folds the oldest turns into short notes the model keeps reading. The fold is a model call (the gauge shows condensing… while it runs), around a second or two with a small model, even on CPU. Either way the transcript on screen and on disk is never touched; only what the model is sent changes.

The engine lifecycle

lilbee loads the model lazily, the way Ollama and LM Studio do: the TUI never waits for the engine, a real llama.cpp server that starts loading in the background the moment the app opens. Ask something before it's ready and the answer bubble carries the load until your answer streams. Nothing freezes and nothing is silently queued.

One engine serves every lilbee process on the machine: open a second TUI, point a coding agent at lilbee mcp, or run lilbee serve alongside, and they all bind to the same loaded models instead of building their own. The engine stops when the last lilbee process exits, so the machine is left clean by default; no VRAM, RAM, or stray processes are held while nothing is running. Two knobs adjust that:

  • Keep engine warm lets the engine outlive lilbee entirely, so the next session's first answer skips the load. Recommended if you relaunch often or use lilbee mainly through coding agents, whose sessions come and go.
  • Engine idle ttl minutes bounds how long idle weights stay in memory in every mode, warm included, five minutes by default (the same idea as Ollama's keep_alive). The memory frees itself after that many idle minutes; a small proxy process stays behind, a few tens of MB and no VRAM. 0 keeps weights loaded until the engine stops.
  • lilbee engine stop frees the shared engine immediately from any terminal, no TUI needed, whichever process started it. It reaches the machine slot and this project root's own overflow engine; a private overflow engine built for a different project root is stopped by running the command from that root.

Both knobs live in the TUI Settings screen, MCP lilbee_settings_set, the HTTP config API, and config.toml. Changing a model takes effect on your next prompt, from whichever surface you changed it. If no one else is using the engine it restarts on the new configuration; if other lilbee processes are serving from it, theirs keeps running and yours either rebinds, when the running engine already serves what you asked for, or starts its own alongside it. The first launch after a reboot is always a cold one.

To stop the engine without opening the TUI:

lilbee engine stop

It reports whether anything was running, frees the GPU immediately, and is safe to run at any time.

Memory

lilbee can remember durable facts about you and standing preferences for how you want answers, and recall them into context on later turns regardless of which conversation they came from. Memory is off by default, so users who don't want it pay nothing and never expose the injection surface. Turn it on once:

/set memory_enabled true

Then save things to remember. A leading pref: stores a standing preference (always recalled); anything else is stored as a fact (recalled by relevance):

/remember pref: keep answers terse, show code first
/remember the Crown Vic manual is my main brake-work reference

Next time you ask a question, the relevant preference and facts are folded into the system prompt before the model answers. Memory is never mixed into the document citations: it shapes the answer but only your actual sources show up under Sources:. Open /memories to list, delete, or share entries.

Memory lives in the active library's data directory, so it follows per-project libraries automatically. A factory /reset clears it; a /rebuild (which only re-indexes documents) leaves it alone. If you switch embedding models, your memories are re-embedded from their stored text during the rebuild, so recall keeps working.

Sharing model. Your own memories are private to you unless you mark them shared. Agent memories (see below) are private to that agent by default and are never folded into your prompt. This asymmetry keeps an agent's notes out of your chat unless you opt in.

Auto-extraction (optional, off by default). With memory_auto_extract on, the TUI runs a small background pass after each answer that saves durable facts and preferences from the exchange, so memory builds up as you chat. The saved memories are recalled like any others; review and prune them anytime in /memories.

/set memory_auto_extract true

The lilbee memory CLI group (add / list / recall / remove) and the REST /api/memories routes operate on the same memories from outside the TUI.

SettingDefaultMeaning
memory_enabledfalseMaster switch for the whole subsystem
memory_auto_extractfalseBackground extraction after each TUI turn
memory_top_k5Max facts recalled per turn
memory_max_distance0.6Recall cutoff (lower is stricter)
memory_token_budget512Token cap on the injected memory block
memory_max_per_owner200Soft cap before oldest memories are evicted

Agent integration

lilbee is also the retrieval backend for AI coding agents. Wire it into any agent that speaks MCP (Claude Code, opencode, Cursor, anything else) and the agent calls lilbee_search / lilbee_add and gets back cited snippets it can quote back. Your files, the embeddings, and the index stay on your computer. See the lilbee-mcp skill for the full MCP tool list and workflows. Non-MCP agents can use the JSON CLI fallback below.

Serving a large agent fleet

One daemon serves many agents at once, and two limits bite before the GPU does. Synchronous work is run off the event loop in a thread pool sized by mcp_tool_threads (40 by default, which is what the async runtime uses when nothing says otherwise); past that, retrieval calls queue while the disk and CPU sit idle, so raise it when a lot of agents search at the same time. Each connected agent also holds a socket, and macOS still defaults to 256 open files (most Linux distributions to 1024), which shows up as connection failures rather than slowness. Raise it in the shell you start the server from:

ulimit -n 4096
lilbee serve

The server logs the current limit at startup when it is low enough to matter.

Agent memory

Agents get their own memory through the lilbee_memory_remember and lilbee_memory_recall MCP tools (memory must be enabled first, e.g. the agent calls lilbee_settings_set({"memory_enabled": true})). An agent's memories are scoped to that agent and private by default: another agent won't see them, and they're never folded into the human TUI's prompt. An agent can recall its own memories plus any of yours you've marked shared.

Each agent's memories are namespaced by an owner like agent:opencode. lilbee derives the agent id from the MCP client name when it can, but pin it explicitly so it stays stable across sessions and clients. Set LILBEE_AGENT_ID in the agent's MCP server config:

{
  "mcpServers": {
    "lilbee": {
      "command": "lilbee",
      "args": ["mcp"],
      "env": { "LILBEE_AGENT_ID": "opencode" }
    }
  }
}

lilbee_memory_remember({"text": "retrieval knobs live in core/config/model.py", "kind": "fact"})
# -> stored under owner agent:opencode
lilbee_memory_recall({"query": "where are retrieval settings"})
# -> returns that note next session; lilbee_search results never mix in memory

JSON CLI fallback

Every lilbee CLI command accepts --json (or -j) before the subcommand for structured output. Use this as the shell-out path for agents that can't speak MCP. The shape mirrors the MCP tools: one JSON object per stdout line, errors return non-zero exit with {"error": "..."}, and distance scores are lower-is-more-relevant. Vectors are stripped from output.

Read (inline, no LLM):

lilbee --json status                           # indexed sources, models, totals
lilbee --json search "query" --top-k 12        # cited chunks (no LLM at query time)
lilbee --json chunks manual.pdf                # inspect how one source was chunked
lilbee --json topics "auth"                    # concept-graph view of a query
lilbee --json model list                       # installed models
lilbee --json model show <ref>                 # catalog + installed metadata for a model
lilbee --json version
lilbee --json self-check                       # runtime + model self-check

Write (LLM calls or long ops):

lilbee --json add ~/docs ~/notes               # register files / dirs, index them in place
lilbee --json add https://example.com/page     # URL becomes a markdown source
lilbee --json sync                             # re-index after edits to the documents directory
lilbee --json rebuild                          # nuke the index and re-ingest everything
lilbee --json remove manual.pdf                # drop chunks (source files are always kept)
lilbee --json remove reports/2024              # a folder: remove everything indexed beneath it
lilbee --json remove '**/*.log'                # a glob: remove every matching source
lilbee --json ask "question"                   # full local RAG (local llama-server fleet or remote backend)
lilbee --json model pull <ref>                 # download a model, streams JSON progress events
lilbee --json model pull <ref> --allow-unsupported  # override the architecture-compat check
lilbee --json model rm <ref>                   # delete an installed model
lilbee --json reset --yes                      # factory reset (destructive, requires --yes)
lilbee --json init [path]                      # create a .lilbee/ in a directory

add is the most common entry point: files, directories, and URLs all go through it, and indexing happens in the same call. Long ops take seconds to minutes; the final JSON includes per-file outcomes and counts.

Wiki (experimental, opt-in):

lilbee --json wiki status                      # page counts + wiki_enabled flag
lilbee --json wiki build                       # generate the topic / entity wiki
lilbee --json wiki update                      # refresh after a sync (full rebuild today)
lilbee --json wiki synthesize                  # cross-source synthesis pages
lilbee --json wiki lint                        # orphans, stale citations, pending drafts
lilbee --json wiki citations <source>          # per-section citation coverage for one source
lilbee --json wiki drafts list                 # pending drafts with drift + faithfulness
lilbee --json wiki drafts diff <slug>          # unified diff between a draft and the live page
lilbee --json wiki drafts accept <slug>        # promote a draft to concepts/ or entities/
lilbee --json wiki drafts reject <slug>        # discard a draft
lilbee --json wiki prune                       # archive stale pages

Two patterns worth knowing:

  • search vs ask. search returns raw chunks without an LLM call. Use it when your agent has its own LLM and just needs context from your files. ask runs lilbee's local RAG end-to-end and returns an answer with sources. Most non-MCP agents want search.
  • Citation rule still applies. Every fact stated from search results must trace back to a chunk's source + line range, exactly as returned. Don't invent.

Output shape:

// lilbee --json search "oil change interval" --top-k 3
{"command": "search", "query": "oil change interval", "results": [
  {"source": "manual.pdf", "chunk": "Change oil every 5,000 miles...", "distance": 0.23, "chunk_type": "raw"}
]}

// lilbee --json status
{"config": {...}, "sources": [{"filename": "manual.pdf", "chunk_count": 42}], "total_chunks": 42}

// lilbee --json model pull <ref>  (streams events, then a final "done" line)
{"event": "progress", "model": "...", "bytes": 12345678, "total": 999999999}
{"event": "done", "model": "...", "installed": true}

Gaps vs MCP. The CLI doesn't expose crawl (non-blocking URL crawling) or per-key settings management. Use add <url> for one-shot URL ingest. For continuous crawling or programmatic settings, the HTTP server exposes both: see the REST API reference.

Caution

Private data and cloud agents

When an agent queries lilbee, retrieved chunks are sent to whatever LLM the agent uses, including cloud-hosted models. If your index contains private, confidential, or sensitive documents, verify two things before connecting an agent:

  1. Check which database is active. Run lilbee status and confirm the data directory is the one you intend the agent to access. lilbee walks up the directory tree to find .lilbee/, so you may be exposing a different project's data than you expect.
  2. Know where your agent sends data. If the agent uses a cloud-hosted model, your document chunks will leave your machine. Use a local model (native GGUF on the managed llama-server fleet, or a local OpenAI-compatible server) if your documents must stay private.

Per-project libraries

lilbee uses a git-like per-project model. Running lilbee init from a project directory creates a .lilbee/ folder there, just like git init creates .git/. Once initialized, every lilbee invocation from that directory (or any subdirectory) automatically uses the local database, both in the TUI and from the CLI:

cd ~/projects/my-engine
lilbee init                  # creates .lilbee/ here
lilbee                       # launches the TUI scoped to this library

If there's no .lilbee/ in the current directory, lilbee walks up the tree looking for one; if none is found, it falls back to the global database at the platform default location (see Data locations). /status in the TUI (or lilbee status from the shell) shows which database is active.

Cloud models

lilbee runs entirely on your machine by default. There are two ways to use cloud models when you want to:

  • Bring your own key, inside lilbee. Install the [litellm] extra and add an API key in /settings → API-Keys, then pick a cloud model from the model bar picker or the Frontier tab in /catalog. The TUI shows a persistent warning whenever a cloud role is active.
  • Pair lilbee with a cloud agent over MCP. lilbee stays the local part: your files, the embeddings, the search index. The agent (Claude Code, opencode, anything that speaks MCP) calls lilbee_search / lilbee_add and gets back cited snippets. See Agent integration.

Either way your files and the index never leave the machine; only the queries and the snippets the model needs to answer cross the wire when you opt in.

Reference for advanced users

Everything from here on is for users who want to script lilbee, run it headless, integrate it with an agent over MCP, or override individual settings without opening the TUI. None of it is required for everyday use; the TUI exposes all of these through pickers, /settings, and slash commands.

CLI commands

Every TUI action is also a CLI command. Indexing and search need an embedding model installed first (the default Nomic v1.5 GGUF is fine):

lilbee model pull nomic-ai/nomic-embed-text-v1.5-GGUF   # required before `lilbee add`
lilbee model browse                                     # pick a chat model interactively

Without an embedding model installed, every lilbee add call reports the file as failed with Model '…' not found in registry.

lilbee add ~/Documents/manual.pdf       # add a file
lilbee add ~/notes/                     # add a directory
lilbee add ~/docs/*.md ~/data/report.pdf
lilbee add manual.pdf --force           # re-index in place (default is skip-if-present)

lilbee search "oil change interval"
lilbee search "oil change interval" --top-k 20

lilbee ask "What is the recommended oil change interval?"
lilbee ask "Explain this" --model qwen3

search only needs the embedding model; ask also needs a chat model.

Manage documents

CommandDescription
lilbee remove manual.pdfRemove from the index (source files are always kept)
lilbee remove reports/2024Remove every document indexed under a folder
lilbee remove '**/*.log'Remove every source matching a glob pattern
lilbee chunks manual.pdfInspect how a document was chunked
lilbee syncRe-index changed files
lilbee rebuildNuke the database and re-ingest everything
lilbee export pages.parquetWrite a per-page text dataset (parquet or jsonl, no vectors)
lilbee import pages.parquetImport a dataset, re-embedding it with the current model
lilbee resetFactory reset. Deletes all documents and data

Wiki

lilbee wiki build                      # build the wiki from the current index
lilbee wiki lint                       # find orphan pages, stale links, pending drafts
lilbee wiki synthesize                 # generate cross-source synthesis pages
lilbee wiki drafts list                # list pending drafts
lilbee wiki drafts accept <slug>       # promote a draft to concepts/ or entities/
lilbee wiki drafts reject <slug>       # discard a draft
lilbee wiki prune                      # move stale pages to archive/

MCP tools mirror the CLI: wiki_list, wiki_read, wiki_synthesize, wiki_lint, wiki_citations, wiki_drafts_list, wiki_drafts_diff, wiki_prune.

Memory

Requires memory enabled (lilbee set memory_enabled true). See Memory for the full model.

lilbee memory add "the project uses rust"        # remember a fact
lilbee memory add "answer tersely" --preference  # remember a standing preference
lilbee memory add "uses rust" --shared           # also expose it to agents
lilbee memory list                               # show stored memories
lilbee memory recall "what language"             # recall facts by relevance
lilbee memory remove <id>                        # delete a memory by id

Sessions

See Sessions. Ids accept any unique prefix.

lilbee sessions list                   # saved conversations, newest first
lilbee sessions show 3f2a              # print a transcript by id prefix
lilbee sessions rename 3f2a "Brake specs"
lilbee sessions delete 3f2a            # asks first; --yes skips the prompt
lilbee --json sessions show 3f2a       # transcript + compaction summary as JSON

Vault and status

lilbee init                            # create .lilbee/ in cwd
lilbee status                          # show active data dir, models, document count
lilbee --global status                 # skip any .lilbee/ and use the platform default
lilbee --data-dir ~/kb status          # use an explicit data dir

Top-level flags

--model / -m, --data-dir / -d, --global / -g, --vision, --vision-timeout, --log-level, --json / -j, --version / -V.

HTTP server

The HTTP server exposes a REST API that any tool or GUI can hit. By default it picks a random port and writes it to <data_dir>/server.port so callers on the same machine can discover it. Start it with lilbee serve:

lilbee serve                           # random port
lilbee serve --port 8080               # fixed port
lilbee serve --host 0.0.0.0            # bind all interfaces (default: 127.0.0.1)

One server runs per data dir: a second lilbee serve against the same --data-dir waits up to fifteen seconds for the first to exit, then stops with an error (exit code 3) instead of competing for the port file and the model engine. A supervisor that manages several data dirs (the Obsidian plugin's shared root) can pass LILBEE_EXCLUSIVE_SCOPE=<dir> to extend the same guarantee across all of them; the refusal then names the data dir the running server is serving. To replace a running server, ask it to stop with POST /api/shutdown (token-authed): it shuts down exactly as an external SIGTERM would, and its locks release the moment it exits.

Every route needs the session token, reads included, GET /api/health among them; the daemon writes the token to server.json (mode 0600) next to the port file, and lilbee agent-config hands it to local clients. The surface covers search (with SSE streaming variants for ask and chat), document lifecycle, crawling, model management, memory (GET/POST/PATCH/DELETE /api/memories, when memory is enabled), saved conversations (/api/sessions: list, read, create, append, rename, delete, and the compaction summary), configuration (including a defaults endpoint that powers per-setting reset), and status/health. The Obsidian plugin uses the /api/source endpoint for vault-aware source retrieval. Interactive REST API docs live at /schema/redoc when the server is running, and the full OpenAPI schema is published at the REST API reference. (Note: this is the HTTP server reference. A Python-library API reference is still being written; for now, the source under src/lilbee/ is the canonical reference.)

Server-specific env vars live in the Server table below.

Running as a service

For tools that talk to lilbee's HTTP REST API all day (the Obsidian plugin, custom GUIs, anything hitting /api/*), your OS launcher can keep the HTTP server warm so requests skip the cold-start. This is the only lilbee surface designed for that pattern; the TUI, lilbee chat, the MCP server, and the rest of the CLI cold-start and exit on every invocation by design.

Pull at least one chat and embedding model first (lilbee model pull <name>). The daemon will start without one, but /api/* requests fail until a model is available. All recipes pin the server to 127.0.0.1:42697.

macOS (Homebrew):

brew services start lilbee

Linux (Arch / AUR, systemd):

systemctl --user enable --now lilbee

On a headless server (no graphical login session), also run loginctl enable-linger $USER so the service survives logout.

NixOS: add the module to your configuration.nix:

{
  imports = [ lilbee.nixosModules.lilbee ];
  services.lilbee.enable = true;
}

Data locations

lilbee resolves the data directory in this order (highest priority first):

PriorityMethodExample
1--data-dir flag or LILBEE_DATA env varlilbee --data-dir ~/my-kb status
2.lilbee/ directory (walks up from cwd)Created by lilbee init
3--global flag (skip .lilbee/, use platform default)lilbee --global status
4Platform defaultSee table below

Platform defaults

PlatformPath
macOS~/Library/Application Support/lilbee/
Linux~/.local/share/lilbee/
Windows%LOCALAPPDATA%/lilbee/

Run lilbee init to create a .lilbee/ directory in your project. It contains documents/ (your indexed files), data/ (the search index), and a .gitignore that excludes derived data.

Config file

Every persisted lilbee setting lives in a single TOML file at <data-dir>/config.toml. For the platform default that's ~/.local/share/lilbee/config.toml; for a project, it's .lilbee/config.toml next to your code. Same file, same shape, regardless of which data dir you're on; the path just follows wherever the data dir is resolved to.

Sections mirror the settings categories in Environment variables below: [general], [retrieval], [chunking], [generation], [server], etc. Example:

# .lilbee/config.toml

[general]
chat_model = "Qwen/Qwen3-8B-GGUF/Qwen3-8B-Q4_K_M.gguf"
theme = "rose-pine"

[retrieval]
top_k = 12
max_distance = 0.6
reranker_model = "Qwen/Qwen3-Reranker-0.6B-GGUF/qwen3-reranker-0.6b.Q8_0.gguf"

[generation]
temperature = 0.2
num_ctx_max = 32768

You don't have to write the file by hand: the TUI's /settings screen and the /set <key> <value> slash command both persist to it, and the equivalent LILBEE_<KEY>=... env vars override individual values at runtime without touching the file.

Environment variables

Every setting has a default that works out of the box, and every value below is also editable from /settings in the TUI. The tables exist for scripted overrides, headless deployments, and CI; you should not normally need to touch them by hand. Tables are grouped from most-commonly-touched to rarely-touched, so you can skim the top and skip the bottom unless you have a specific reason.

Common settings

The ones most users set at least once.

VariableDefaultDescription
LILBEE_DATA(platform default)Data directory path. Overridden by --data-dir or a .lilbee/ vault walked up from cwd
LILBEE_CHAT_MODELQwen/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.ggufChat model. Native GGUF by default; with pip install --pre 'lilbee[litellm]' (or uv tool install --prerelease=allow 'lilbee[litellm]'), any remote name the SDK backend understands
LILBEE_EMBEDDING_MODELnomic-ai/nomic-embed-text-v1.5-GGUF/nomic-embed-text-v1.5.Q4_K_M.ggufEmbedding model. Changing this requires lilbee rebuild
LILBEE_VISION_MODEL(none)Vision OCR model. When set, takes precedence over Tesseract on scanned PDFs and images
LILBEE_OCR_TIMEOUT300Per-page vision OCR timeout in seconds (0 = no limit)
LILBEE_LOG_LEVELWARNINGLogging level (DEBUG, INFO, WARNING, ERROR)
LILBEE_RAG_SYSTEM_PROMPT(built-in)Custom system prompt for RAG (search-mode) answers
LILBEE_GENERAL_SYSTEM_PROMPT(built-in)Custom system prompt for chat-mode (ungrounded) answers
LILBEE_SHOW_REASONINGfalseShow the model's <think> reasoning tokens in chat output. Useful with Qwen3, DeepSeek-R1, and other reasoning models
LILBEE_HF_TOKEN(none)HuggingFace access token. Avoids the unauthenticated download rate limit and unlocks gated repos. Same token can be persisted via the System tab in /settings (stored in plain text at config.toml). HF_TOKEN env var also accepted

Retrieval tuning

Reach for these when search quality matters. Defaults are solid; tune only if something feels off.

VariableDefaultDescription
LILBEE_TOP_K12Number of retrieval results returned
LILBEE_MAX_DISTANCE0.75Cosine distance cutoff for rows without keyword support. Lower = stricter filtering. 0 disables filtering
LILBEE_MMR_LAMBDA0.5Relevance vs. diversity balance on the vector-only fallback path (no effect on the default hybrid search)
LILBEE_DIVERSITY_MAX_PER_SOURCE5Max chunks from a single source document in the top-K. Prevents one big file from dominating results
LILBEE_QUERY_EXPANSION_COUNT3LLM-generated query variants per search. 0 disables expansion entirely for faster queries
LILBEE_RERANKER_MODEL(none)GGUF cross-encoder reranker for a precision pass over top results. See Cross-encoder reranking
LILBEE_RERANK_CANDIDATES60Candidates to rerank when a reranker is configured
LILBEE_RERANK_BLENDtrueBlend reranker scores with retrieval fusion; off = the reranker's own ordering stands
LILBEE_HYDEfalseEnable Hypothetical Document Embeddings: an LLM drafts a hypothetical answer, that's embedded, and results are merged with the original query's. Adds ~500 ms per query; helps on vague questions
LILBEE_HYDE_WEIGHT0.7How much to trust HyDE results relative to the direct query (0.0-1.0)
LILBEE_ADAPTIVE_THRESHOLDfalseWiden the distance threshold step by step when too few results pass, on the vector-only fallback path (no effect on the default hybrid search)
LILBEE_ADAPTIVE_THRESHOLD_STEP0.2How much to widen per step when adaptive threshold triggers
LILBEE_TEMPORAL_FILTERINGtrueWhen the query contains temporal cues ("recent", "last week"), filter results by document date and sort by recency
LILBEE_MAX_CONTEXT_SOURCES8Max chunks included in the LLM's RAG context. Raise for more coverage, lower for shorter prompts
LILBEE_EXPANSION_SHORT_QUERY_TOKENS2Queries this short (in tokens) skip LLM query expansion
LILBEE_ANN_INDEX_THRESHOLD50000Chunk count above which sync builds the ANN vector index
LILBEE_MAX_REASONING_CHARS64000Cap on a reasoning model's thinking output per answer
LILBEE_MEMORY_DEDUP_DISTANCE0.05Cosine distance under which a new memory is a duplicate
LILBEE_WIKI_CLUSTERERembeddingWiki topic clusterer backend (embedding or graph)
LILBEE_WIKI_CLUSTERER_K0Fixed wiki cluster count (0 = choose automatically)
LILBEE_CHAT_MODEsearchDefault answer mode: search (grounded) or chat (ungrounded)

Ingestion and chunking

How documents become chunks. Changes here require lilbee rebuild to take effect on already-indexed material.

VariableDefaultDescription
LILBEE_CHUNK_SIZE512Target tokens per chunk
LILBEE_CHUNK_OVERLAP100Overlap tokens between adjacent chunks
LILBEE_MAX_EMBED_CHARS2000Max characters per chunk passed to the embedder
LILBEE_SEMANTIC_CHUNKINGfalseExperimental topic-aware chunking. See Semantic chunking
LILBEE_TOPIC_THRESHOLD0.75Cosine boundary threshold for semantic chunking (lower = more splits)
LILBEE_EMBEDDING_DIM768Embedding dimensionality. Must match the embedding model
LILBEE_EMBED_REPLICAS1Embedding servers to run in parallel, one per spare GPU, for large-scale ingest. Capped at runtime by the GPUs with room after the chat model is placed
LILBEE_VISION_REPLICAS1Vision OCR servers to run in parallel, one per spare GPU, for large-scale ingest. Same runtime cap as LILBEE_EMBED_REPLICAS
LILBEE_VISION_OCR_MAX_TOKENS4096Hard cap on tokens generated per OCR page. A real page is well under this; the cap bounds runaway repetition loops
LILBEE_VISION_OCR_CONCURRENCY4Pages OCR'd concurrently, and the vision server's continuous-batching slots. Each slot adds KV cache, so lower it on small GPUs

Generation

LLM output shape. lilbee sets conservative defaults below; the model's own defaults apply only when a value is explicitly unset in code or config.

VariableDefaultDescription
LILBEE_TEMPERATURE0.1Sampling temperature
LILBEE_TOP_P0.9Nucleus sampling threshold
LILBEE_TOP_K_SAMPLING40Top-k sampling
LILBEE_REPEAT_PENALTY1.1Repetition penalty
LILBEE_NUM_CTX(auto)Context window size. Empty = sized automatically (aims for LILBEE_CHAT_N_CTX_TARGET, ceiling at LILBEE_NUM_CTX_MAX or the model's training_ctx). Set explicitly to lock a specific value
LILBEE_CHAT_N_CTX_TARGET(auto)Target context size for the dynamic picker, scaled by total host RAM: <16 GiB → 8192, 16-32 GiB → 12288, 32-64 GiB → 16384, ≥64 GiB → 24576. 8192 is the floor on smaller hosts. The picker still clamps the result to the model's training context and available memory at worker start. Set explicitly to override
LILBEE_NUM_CTX_MAX(auto)Explicit ceiling for the dynamic context picker. Empty = use the model's training_ctx from GGUF metadata as the only ceiling. Set to cap below training_ctx on memory-constrained hosts
LILBEE_FLASH_ATTENTION(auto)Flash attention for the chat server. Empty/auto enables it; 1/true/on forces on; 0/false/off disables. Resolves the padding V cache to 1024 warning on models with uneven per-layer V dims
LILBEE_KV_CACHE_TYPEq8_0KV cache element type: f16, f32, q8_0, q4_0. q8_0 (default) halves KV memory vs f16 with no measurable chat-quality loss; q4_0 quarters it with a small quality cost. Quantized variants require flash attention to be enabled
LILBEE_N_GPU_LAYERS(auto)Layers to offload to GPU. Empty/auto = all (recommended), cpu = none, integer = partial offload for tight VRAM
LILBEE_CPU_MOEfalseKeep a mixture-of-experts model's expert weights in system memory so it fits a smaller GPU. Only the ~3B active parameters stay on the card, which is what lets an 80B MoE run on a single consumer GPU. No effect on dense models, which have no expert tensors
LILBEE_N_CPU_MOE(none)Offload only the first N layers' experts instead of all of them. Takes precedence over LILBEE_CPU_MOE; a smaller N keeps more of the model on the GPU, so tune it up until the model fits
LILBEE_SEED(model default)Random seed for reproducibility
LILBEE_LLAMA_SERVER_PATH(bundled)Path to a llama-server binary; when set it is always used, even if the lilbee-engine wheel is installed. Empty = the bundled wheel's binary, else one found on PATH

Server

Only relevant when running the HTTP server.

VariableDefaultDescription
LILBEE_SERVER_HOST127.0.0.1Bind address
LILBEE_SERVER_PORTrandomPort (overridden by --port)
LILBEE_EXCLUSIVE_SCOPE(none)Directory that at most one server may serve at a time, on top of the per-data-dir lock. A second lilbee serve pointed at the same scope waits up to fifteen seconds, then exits with an error naming the data dir the running server is serving. Managed supervisors set this (the Obsidian plugin passes its shared root)
LILBEE_ENGINE_DIR(platform default)Where the shared engine slot lives. Every lilbee on the machine binds the engine recorded here rather than starting its own, so pointing two installs at the same path makes them share one fleet. Set this when the default lands on a filesystem without working file locks, where lilbee declines to share and logs that it is doing so
LILBEE_CORS_ORIGINS(none)Comma-separated list of extra allowed CORS origins, e.g. https://my-app.com. Additive; the default regex below still applies
LILBEE_CORS_ORIGIN_REGEX(see usage)Regex for allowed origins. Default matches app://obsidian.md, capacitor://localhost, and any http(s)://localhost, 127.0.0.1, or [::1] with any port. Set to ^$ to opt out and rely solely on LILBEE_CORS_ORIGINS
LILBEE_ALLOW_HTTP_PLACEMENTfalseAllow PUT/DELETE /api/placement to apply or clear GPU placement over HTTP. Off by default because applying placement restarts the fleet's moved roles, which is unsafe across concurrent clients. Turn it on only for a single-client or owned deployment (the Obsidian plugin's managed server, or a personally-owned pod where you run lilbee serve yourself)

Wiki tuning (experimental)

Only relevant if you run lilbee wiki build.

VariableDefaultDescription
LILBEE_WIKI_INGEST_UPDATE_CAP20Max changed sources processed by incremental wiki updates during lilbee sync. Prevents a big re-ingest from churning concepts
LILBEE_WIKI_CONCEPT_MAX_CHUNKS_PER_PAGE25Top-K chunks behind each wiki page section

Advanced

Rarely touched. Defaults derived from published IR research; there's usually a reason the defaults are the defaults.

VariableDefaultDescription
LILBEE_EXPANSION_SKIP_THRESHOLD0.8BM25 confidence above which query expansion is skipped; confidence is s / (s + 5), so 0.8 = raw score 20
LILBEE_EXPANSION_SKIP_GAP0.15Minimum relative raw-score gap between top-1 and top-2, (top - second) / top, for expansion to skip
LILBEE_EXPANSION_GUARDRAILStrueFilter expansion variants whose embedding drifts too far from the original query
LILBEE_EXPANSION_SIMILARITY_THRESHOLD0.5Minimum query-variant cosine similarity to survive the guardrail
LILBEE_CANDIDATE_MULTIPLIER3Vector-only candidate pool as a multiple of top_k, feeding MMR reranking
LILBEE_ENTITY_EXTRACTIONfalseExtract typed entities for exact count answers; fully automatic at sync (schema induced from the corpus, stored inside the index, and re-induced as the library grows)
LILBEE_MIN_RELEVANCE_SCORE0.0Abstention floor against the canonical [0, 1] relevance score; when every result falls below it, ask refuses instead of answering from noise
LILBEE_HISTORY_REWRITEtrueCondense follow-up questions into standalone retrieval queries using chat history
LILBEE_INTENT_ROUTINGtrueRoute document-name lookups to exact retrieval and count questions to a full-corpus scan
LILBEE_INTENT_LLMfalseClassify count questions with the chat model when the fast patterns miss; covers phrasing variants and other languages at the cost of one short LLM call on those turns

Optional extras

lilbee works out of the box with its managed llama-server fleet for local inference. These optional extras add capabilities that require heavier dependencies:

# pip
pip install --pre 'lilbee[graph]'      # concept graph: topic clustering + search boosting
pip install --pre 'lilbee[crawler]'    # web crawling: index websites alongside local docs
pip install --pre 'lilbee[litellm]'    # remote providers: connect to any SDK-backed provider

# uv tool
uv tool install --prerelease=allow 'lilbee[graph]'
uv tool install --prerelease=allow 'lilbee[crawler]'
uv tool install --prerelease=allow 'lilbee[litellm]'

Install multiple at once:

pip install --pre 'lilbee[graph,crawler,litellm]'
uv tool install --prerelease=allow 'lilbee[graph,crawler,litellm]'

NVIDIA users: the default Vulkan build works, but the CUDA flavour is faster and dodges the Vulkan-loader crash that affects NVIDIA-on-Windows setups. Same lilbee command, links straight against your NVIDIA driver:

pip install --pre lilbee --extra-index-url https://lilbee.sh/cu125/
brew install tobocop2/lilbee/lilbee-cuda
paru -S lilbee-cuda
nix run github:tobocop2/lilbee#lilbee-cuda

The lilbee-cuda AUR package conflicts with lilbee and provides it, so paru -S lilbee-cuda swaps automatically. On Homebrew, run brew uninstall lilbee first because the two formulas both ship a lilbee binary. For older drivers swap cu125 for cu124 or cu121 (run nvidia-smi to see which matches). cu121 and cu124 are wheel-index and direct-download only; cu125 is the variant fanned out to the package managers.

While 0.6.66 is in beta, the --pre flag (or uv's --prerelease=allow) is required on every install.

Cross-encoder reranking is built in (no extra required); see Cross-encoder reranking below.

Concept graph

Builds a topic map of your documents at index time. Related concepts are linked in a co-occurrence graph, which is used to boost search results and expand queries with related terms, all without extra LLM calls.

What it does: Extracts noun phrases from every chunk using spaCy, computes PMI co-occurrence weights between concepts, and clusters them with the Leiden algorithm. At search time, queries are expanded with graph neighbors and results overlapping query concepts get a relevance boost.

When to use it: Large corpora (100+ documents) where the same topics appear across multiple files. The graph helps surface connections that pure vector similarity misses. For example, finding "deployment" documents when searching for "CI/CD" because those concepts co-occur frequently.

Install: pip install --pre 'lilbee[graph]' or uv tool install --prerelease=allow 'lilbee[graph]'

Configuration:

export LILBEE_CONCEPT_GRAPH=true              # enable (default: true when deps installed)
export LILBEE_CONCEPT_BOOST_WEIGHT=0.3        # how much concept overlap matters (0.0-1.0)
export LILBEE_CONCEPT_MAX_PER_CHUNK=5         # max concepts extracted per chunk (default)

The graph is built automatically during lilbee sync. No extra commands needed; search results are boosted transparently.

Based on: Microsoft Research's LazyGraphRAG technique, Church & Hanks 1990 (PMI), Traag et al. 2019 (Leiden).

Web crawling

Index web pages alongside your local documents. Crawl single pages or follow links recursively.

What it does: Fetches web pages using a headless browser (Playwright), extracts markdown content, and indexes it. Supports recursive crawling with configurable depth, concurrent fetching, live progress, cancel, per-domain rate-limit + retries on HTTP 429/503, and SSRF protection against internal network access.

When to use it: When your library spans both local files and web content such as documentation sites, wikis, or internal tools. Crawled content is hash-tracked so re-crawling only re-indexes changed pages.

Install: pip install --pre 'lilbee[crawler]' or uv tool install --prerelease=allow 'lilbee[crawler]'

Usage:

# Single page (no --crawl)
lilbee add https://docs.example.com/guide

# Whole-site crawl (recursive, unbounded by default)
lilbee add https://docs.example.com --crawl

# Cap depth or page count
lilbee add https://docs.example.com --crawl --depth 2 --max-pages 200

# Multiple URLs
lilbee add https://docs.example.com https://wiki.example.com

Also available via MCP (crawl), REST API (POST /api/crawl), and TUI (/crawl).

Configuration (all optional):

# Global ceilings. Unset = no cap. Explicit --depth/--max-pages always win.
export LILBEE_CRAWL_MAX_DEPTH=3          # cap link-following depth
export LILBEE_CRAWL_MAX_PAGES=1000       # cap total pages

# Pacing within a single crawl.
export LILBEE_CRAWL_MEAN_DELAY=0.5       # seconds between requests
export LILBEE_CRAWL_MAX_DELAY_RANGE=0.5  # random jitter on top
export LILBEE_CRAWL_CONCURRENT_REQUESTS=3

# Per-domain rate-limit + retries on HTTP 429/503.
export LILBEE_CRAWL_RETRY_ON_RATE_LIMIT=true
export LILBEE_CRAWL_RETRY_BASE_DELAY_MIN=1.0
export LILBEE_CRAWL_RETRY_BASE_DELAY_MAX=3.0
export LILBEE_CRAWL_RETRY_MAX_BACKOFF=30.0
export LILBEE_CRAWL_RETRY_MAX_ATTEMPTS=3

# Other.
export LILBEE_CRAWL_TIMEOUT=30           # per-page timeout (seconds)
export LILBEE_CRAWL_MAX_CONCURRENT=0     # 0 = CPU count (top-level concurrency)
export LILBEE_CRAWL_SYNC_INTERVAL=30     # seconds between periodic syncs during crawl

Remote providers (SDK backend)

Connect to hosted or local OpenAI-compatible LLM backends alongside lilbee's managed local llama-server engine.

What it does: Routes chat and embedding calls to any provider reachable via the SDK backend. The routing provider automatically detects which models are available locally vs. remotely and routes each call to the right backend.

When to use it: When you want a frontier model for chat while keeping embeddings local for privacy, or to surface models from a local OpenAI-compatible daemon alongside lilbee's native GGUF models.

Install: pip install --pre 'lilbee[litellm]' or uv tool install --prerelease=allow 'lilbee[litellm]'.

Configuration:

export LILBEE_LLM_PROVIDER=auto          # "auto" routes between local and remote
export LILBEE_OLLAMA_BASE_URL=http://localhost:11434  # Ollama URL (blank = default)
export LILBEE_LM_STUDIO_BASE_URL=http://localhost:1234/v1  # LM Studio URL (blank = default)
export LILBEE_LLM_API_KEY=sk-...         # API key for your provider
export LILBEE_CHAT_MODEL=your-model      # any remotely-supported model name

Provider options: auto (default; native GGUF models run on the local managed llama-server fleet, remote model names route to the SDK backend) and remote (everything goes to an external OpenAI-compatible endpoint).

Tuning GPU placement

By default lilbee decides which GPUs each model goes on automatically. It bin-packs all four roles (chat, embed, rerank, vision) across your available GPUs and tensor-splits anything too large for one card.

If you want to pin specific models to specific cards, you can set a placement spec. The spec is a JSON object with one entry per role you want to control:

{
  "chat":   { "devices": [0, 1], "tensor_split": [1, 1] },
  "embed":  { "devices": [2] },
  "rerank": { "devices": [2] },
  "vision": { "devices": [3] }
}

Each role takes devices (GPU indices), an optional tensor_split list for spreading a single model across cards, and an optional replicas count for the embed and vision roles. Omit a role and the auto planner handles it.

To see what the auto planner would assign (without changing anything):

lilbee placement preview

To apply a spec from a file:

lilbee placement set --spec placement.json

To see the current spec (or confirm auto is active):

lilbee placement show

To go back to automatic:

lilbee placement clear

The same operations are available in the TUI under the Placement screen and over MCP (set_placement, clear_placement). Applying or clearing placement rebuilds the shared fleet, so over HTTP PUT/DELETE /api/placement are refused by default. Set allow_http_placement (or LILBEE_ALLOW_HTTP_PLACEMENT=1) to enable them on a single-client or owned deployment, such as the Obsidian plugin's managed server or a personally-owned pod where you run lilbee serve yourself. The spec persists across restarts.

If a pinned placement no longer fits the card it names (after a hardware change, for example), lilbee surfaces an error naming the card rather than starting in a broken state. lilbee placement clear returns to automatic placement in that case.

Cross-encoder reranking

Built-in. Re-scores retrieval candidates with a cross-encoder for precision on the top results. Unlike the extras above, no extra install is required; reranking is off by default and turns on as soon as you set LILBEE_RERANKER_MODEL (or pick a reranker from /settings).

What it does: After hybrid search (BM25 and vector arms fused into one canonical relevance score) returns candidates, a GGUF cross-encoder scores each (query, chunk) pair and results are blended with position-aware weights. Top-ranked candidates keep more of the original ranking; lower-ranked candidates trust the reranker more.

When to use it: When you need high-precision answers and are willing to trade roughly 200 to 500 ms per query. Most useful with large candidate sets where top-5 ordering matters.

Configuration:

export LILBEE_RERANKER_MODEL="bge-reranker-v2-m3"   # any GGUF reranker
export LILBEE_RERANK_CANDIDATES=20                  # how many candidates to rerank

Without a reranker set, fused hybrid search already provides good results for most use cases.

Based on: Nogueira & Cho 2019 (Passage Re-ranking with BERT), Burges et al. 2005 (Learning to Rank).

Semantic chunking

Experimental. Off by default. lilbee ships with two chunking strategies; which one serves you depends on what you're indexing.

Fixed-size (default). Breaks documents into roughly equal token windows with overlap. Fast, deterministic, works well on code, reference manuals, user guides, API specs, and anything with clear structural boundaries. The assumption is that each chunk only needs to be coherent enough for retrieval, and the model will handle the rest from a small window of context.

Semantic (experimental). Uses embedding similarity to detect topic boundaries and splits there instead of at fixed sizes. Each chunk tends to represent one coherent thought rather than an arbitrary slice through one. The benefit shows up on prose-heavy material: novels, essays, long-form research papers, interview transcripts, qualitative research notes, anything where an argument develops across paragraphs. When you ask a question, the retrieved chunk is more likely to contain the full section that matches rather than the first half of it plus unrelated setup.

Trade-off: Enabling semantic chunking triggers a one-time download of kreuzberg's ONNX embedding model (separate from the chunk-to-vector embedder) and runs roughly 9x more downstream embedding calls during indexing. Indexing takes longer; retrieval latency is unchanged.

How to enable it

Three equivalent paths:

# Environment variable
export LILBEE_SEMANTIC_CHUNKING=true

# TUI /set command (interactive)
/set semantic_chunking true

# config.toml in your .lilbee/ vault
[general]
semantic_chunking = true

After enabling, run lilbee rebuild so existing documents are re-chunked under the new strategy. New documents added from that point use semantic chunking automatically.

Tuning

export LILBEE_TOPIC_THRESHOLD=0.75   # cosine threshold for topic boundaries (0.0-1.0)

OCR

OCR is how lilbee gets text out of scanned PDFs and images. It's one step in indexing, not a substitute for it: however the text comes out (a native parser, Tesseract, or a vision model), it still gets embedded, so you still need an embedding model installed.

For PDFs without embedded text, lilbee supports two OCR backends. When a vision model is configured, it takes precedence.

TesseractVision model
OutputPlain textStructured markdown (tables, headings)
Retrieval qualityFragments lose contextChunks preserve semantic boundaries
InstallSystem package (brew/apt)Native GGUF via the built-in mtmd backend, or any vision model reachable via the SDK backend (pip install --pre 'lilbee[litellm]' / uv tool install --prerelease=allow 'lilbee[litellm]')
Best forSimple text-only scansTables, multi-column layouts, formatted docs

See model benchmarks for detailed comparisons.

Tesseract

Tesseract is used automatically when no vision model is configured. No flags needed.

brew install tesseract          # macOS
sudo apt install tesseract-ocr  # Ubuntu/Debian

Vision models

lilbee runs vision OCR in one of two ways:

  1. Local vision model. Point LILBEE_VISION_MODEL at a GGUF vision model (e.g. lightonocr) and lilbee serves it on llama-server with an --mmproj projector. This is the recommended path and supports an SSE heartbeat for long scans.
  2. Remote vision model. With pip install --pre 'lilbee[litellm]' (or uv tool install --prerelease=allow 'lilbee[litellm]'), set the vision model to any remote name your SDK backend understands. lilbee will route vision calls accordingly.
lilbee add report.pdf --vision                # prompts for model if none set
lilbee add report.pdf --vision-timeout 30     # per-page timeout (default: 120s, 0 = no limit)
export LILBEE_VISION_MODEL=lightonocr         # persist across runs (GGUF via mtmd)

Pick or change a vision model interactively via /settings or /setup in the TUI; the selection is saved to config.toml and persists across sessions.