Jude

March 7, 2026 · View on GitHub

A local AI study assistant for Jewish texts. Ask questions about Torah, Talmud, Halacha, Kabbalah, Midrash, and more. Runs fully offline.

How it works

Each question passes through a five-stage pipeline:

Router — classifies the question by tier (halacha, narrative, debate, mystical) and depth, generates search queries. For halachic questions, a second fast classifier identifies the specific topic (Shabbat, Divorce, Kashrut, etc.) and maps it to its canonical source hierarchy.
Retrieval — queries a ChromaDB vector database of 288,737 Sefaria passages. Halachic questions use dual retrieval: a primary pass fetches sources strictly from the topic's canonical hierarchy (Torah → Mishnah → Talmud → Rambam → Shulchan Arukh), then a secondary pass sweeps the broader corpus for contextual material.
Filter — an LLM pass drops irrelevant sources
Synthesis — streams a cited answer tuned to the question type. Primary sources are labeled and cited first in the canonical halachic order.
Tools (optional, requires llama3.2+) — the synthesiser can call retrieve_more, lookup_ref, and find_parallel mid-generation to pull additional passages as needed.

Results are streamed live to the browser. Every source card links back to Sefaria.

Three modes: Q&A (default), Study (pre-loads a deep source pool for sustained exploration), Sources Only (skip synthesis, browse raw passages).

Hebrew input is detected automatically; search queries are translated to English before retrieval so the full corpus is always searchable.

Models run via Ollama by default. You can also point Jude at any OpenAI-compatible cloud provider (OpenAI, Groq, Together, etc.) using a .env file — see Configuration below.

Halachic topic routing

When a question is identified as halachic, Jude detects the specific topic and enforces a canonical source hierarchy for retrieval and citation:

Topic	Seder	Canonical sources
Shabbat	Moed	Exodus → Mishnah Shabbat → Shabbat → Mishneh Torah, Sabbath → Shulchan Arukh OC
Eruvin	Moed	Mishnah Eruvin → Eruvin → Mishneh Torah, Eruvin
Pesach	Moed	Exodus → Mishnah Pesachim → Pesachim → Mishneh Torah, Leavened and Unleavened Bread
Yom Tov	Moed	Mishnah Beitzah → Beitzah → Mishneh Torah, Rest on a Holiday
Rosh Hashanah	Moed	Mishnah Rosh Hashanah → Rosh Hashanah → Mishneh Torah, Shofar...
Yom Kippur	Moed	Leviticus → Mishnah Yoma → Yoma → Mishneh Torah, Rest on the Tenth of Tishrei
Sukkot	Moed	Leviticus → Mishnah Sukkah → Sukkah → Mishneh Torah, Shofar...
Purim / Megillah	Moed	Mishnah Megillah → Megillah → Mishneh Torah, Scroll of Esther...
Fast Days	Moed	Mishnah Ta'anit → Ta'anit → Mishneh Torah, Fasts
Marriage	Nashim	Deuteronomy → Mishnah Kiddushin → Kiddushin → Mishneh Torah, Marriage → SA Even HaEzer
Divorce	Nashim	Deuteronomy → Mishnah Gittin → Gittin → Mishneh Torah, Divorce → SA Even HaEzer
Niddah	Tahorot	Leviticus → Mishnah Niddah → Niddah → Mishneh Torah, Forbidden Intercourse
Kashrut	Kodashim	Leviticus → Mishnah Chullin → Chullin → Kitzur SA → SA Yoreh De'ah
Prayer	Zeraim	Mishnah Berakhot → Berakhot → Mishneh Torah, Prayer... → SA Orach Chayim
Blessings	Zeraim	Mishnah Berakhot → Berakhot → Kitzur Shulchan Arukh
Torts & Damages	Nezikin	Exodus → Mishnah Bava Kamma → Bava Kamma → Mishneh Torah, Laws of Wounding...
Loans & Contracts	Nezikin	Mishnah Bava Metzia → Bava Metzia → Mishneh Torah, Creditor and Debtor
Vows / Nedarim	Nashim	Numbers → Mishnah Nedarim → Nedarim → Mishneh Torah, Vows
Temple Sacrifices	Kodashim	Leviticus → Mishnah Zevachim → Zevachim → Mishneh Torah, Sacrificial Procedure

Each topic also carries a related_books list — topic-adjacent tractates used for the secondary retrieval pass (e.g. Shabbat → Eruvin + Beitzah; Divorce → Ketubot + Yevamot). This ensures secondary sources come from halachically relevant tractates rather than a generic corpus sweep.

The topic badge (e.g. 📚 Shabbat · Seder Moed) is shown in the answer header. Source cards are grouped into Primary Sources (gold border, ⚖ badge) and Secondary Sources.

Setup

1. Prerequisites

Python 3.11+
Ollama installed and running
The sefaria_rag_markdown/ corpus and chroma_db/ index (see below)

Pull the required models:

ollama pull llama3.2
ollama pull nomic-embed-text

llama3.2 handles all pipeline roles — routing, filtering, summarisation, and tool-calling synthesis. No separate model is needed.

2. Install dependencies

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

3. Download the data

The corpus and vector index are hosted on Hugging Face (too large for GitHub):

pip install huggingface_hub
python - <<'EOF'
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="RockyCo/jude-judaic-data",
    repo_type="dataset",
    local_dir=".",
)
EOF

This downloads sefaria_rag_markdown/ (~1.2 GB) and chroma_db/ (~2.0 GB) into the project root.

Or rebuild the index yourself from the corpus only:

python -m backend.indexer

4. Start the server

Option A — server + terminal UI together (recommended):

python run.py

This starts the FastAPI server in the background and opens the terminal UI in the same window. Server logs are written to server.log. Press Ctrl+C in the terminal UI to stop everything.

Option B — server only (web UI):

python run.py --web-only
# or the classic:
uvicorn backend.main:app --reload --port 8000

Open http://localhost:8000.

Option C — terminal UI against an already-running server:

python tui.py
# or with a custom host/port:
python tui.py --host localhost --port 8000

`run.py` flags

Flag	Default	Description
`--port`	8000	Port for server and TUI
`--host`	127.0.0.1	Bind host for the server
`--web-only`	off	Start server only, no terminal UI
`--tui-only`	off	Skip starting server (connect to existing)
`--reload`	off	Pass `--reload` to uvicorn (dev mode)

Configuration

Dynamic Cloud Fallback

Jude is designed and tuned to use a highly reliable fallback cascade for synthesis. If a primary API key hits a rate limit or times out, the backend seamlessly routes the request to the next available provider in the chain without dropping the user's query.

Copy .env.example to .env and fill in your credentials to enable cloud models:

cp .env.example .env

# .env
GEMINI_API_KEY=AIzaSy...     # Priority 1: Google Gemini (fastest, high quality)
LLMOD_API_KEY=sk-...         # Priority 2: LLMod RPRTHPB-gpt-5-mini
# If both fail or are missing, Priority 3: Ollama (Local) takes over

Note: The Gemini free tier has a strict 5 queries per minute rate limit. This fallback system allows Jude to handle sudden concurrency spikes by silently spilling over to LLMod and then to local execution.

Priority Cascade

Gemini 2.5 Flash / gpt-4o scale (if GEMINI_API_KEY is present)
LLMod RPRTHPB-gpt-5-mini (if LLMOD_API_KEY is present)
Local Ollama llama3.2 (Default absolute fallback / Offline mode)

Per-role overriding

While synthesis uses the fallback chain to prevent timeouts, you can lock specific functional roles (like routing or filtering) to ALWAYS use the local Ollama instance explicitly, saving cloud tokens on easy JSON tasks:

# .env — typical hybrid routing
ROUTER_PROVIDER=ollama      # "ollama" | "cloud"
FILTER_PROVIDER=ollama
SUMMARY_PROVIDER=ollama
# Synthesis/Tools will still use the dynamic Gemini -> LLMod -> Ollama cascade

You can also completely override the cascade by supplying a direct OpenAI-compatible endpoint and key for any role:

# Force the router to use Groq specifically:
ROUTER_API_KEY=gsk_...
ROUTER_BASE_URL=https://api.groq.com/openai/v1
ROUTER_MODEL=llama-3.3-70b-versatile

Model configuration guide

Each pipeline step has different requirements:

Role	Task	Quality need	Local (llama3.2)	Cloud options
`ROUTER_MODEL`	Classify tier, depth, search queries (JSON ~150 tok)	Medium	2–7 s	gpt-4o-mini (~0.5 s), Groq llama3 (~0.3 s)
`FILTER_MODEL`	Score each source keep/drop (JSON ~200 tok)	Medium, fail-open	4–5 s	gpt-4o-mini, Groq llama3
`SUMMARY_MODEL`	Compress chat history (2-4 sentences)	Low	1–2 s	any
`SYNTH_MODEL`	Full cited structured answer (500–2000 tok)	High	30–90 s	gpt-4o (~20 s), claude-3-5-sonnet, gpt-4o-mini
`TOOLS_MODEL`	Tool selection + full answer (needs function calling)	High	30–90 s	same as SYNTH

Recommended configurations:

Profile	Router / Filter / Summary	Synthesis	Total latency	Cost
Full local	Ollama llama3.2	Ollama llama3.2	30–90 s	Free
Hybrid balanced (default)	Ollama llama3.2	`gpt-4o-mini` cloud	20–50 s	Low (~$0.001/query)
Hybrid quality	Ollama llama3.2	`gpt-4o` cloud	15–40 s	Medium
Groq accelerated	Groq llama-3.3-70b	`gpt-4o` cloud	5–20 s	Medium
Full cloud	`gpt-4o-mini` cloud	`gpt-4o` cloud	40–120 s	High

The default .env.example ships with the Hybrid balanced profile: Ollama for fast tasks, cloud model for synthesis quality.

Jude supports lightweight name-based user sessions. On first visit a login modal asks for a name. The name is stored in localStorage and sent with every request so each user sees only their own chat history. There is no password or server-side authentication — it is purely a convenience for shared local deployments.

To switch users, click your name in the top-right corner of the interface.

Disk space

Item	Size	Notes
`llama3.2` (Ollama)	2.0 GB	Router, filter, summarisation, tool-calling synthesis
`nomic-embed-text` (Ollama)	274 MB	Embeddings — always required
`chroma_db/`	2.0 GB	Vector index — required at runtime
`sefaria_rag_markdown/`	1.2 GB	Source corpus — required for Hebrew text on source cards and re-indexing
Total	~5.5 GB	+(cloud synthesis model, no local storage needed)

Terminal UI

The terminal UI connects to the same /api/chat SSE endpoint as the browser and supports the same three modes (Q&A, Study, Sources Only).

Your name: Alice

Mode: qa  ·  Type /help for commands

You: What is the biblical source for lighting Shabbat candles?
  Retrieving sources…
  Filtering sources…

Jude: The primary biblical basis for Shabbat candle lighting derives from…

  Shabbat 25b        The Sages derived the obligation…
  Mishnah Shabbat 2  These are the wicks…

You: /mode study
Mode set to 'study'. New chat started.

You: /quit
Goodbye.

Available commands:

Command	Action
`/new`	Start a fresh chat
`/mode qa`	Switch to Q&A mode
`/mode study`	Switch to Study mode
`/mode sources`	Switch to Sources-only mode
`/help`	Show command reference
`/quit`	Exit

Project structure

backend/
  main.py          # FastAPI server, SSE streaming
  llm.py           # LLM provider abstraction (Ollama + OpenAI-compatible)
  router.py        # Question routing, halachic topic classification
  retriever.py     # ChromaDB vector search, dual retrieval, reranking
  filter.py        # Post-retrieval relevance filter
  synthesiser.py   # Answer generation (streaming + tool-calling)
  tools.py         # Agentic tool definitions and dispatch
  chat_store.py    # Chat history (JSON) + session caches
  guard.py         # Prompt injection defense
  indexer.py       # One-time index builder
frontend/
  index.html       # Single-file web UI
tui.py             # Terminal UI client
run.py             # Launcher: starts server + terminal UI together
chroma_db/         # Vector index (required at runtime)
sefaria_rag_markdown/  # Source corpus (required for Hebrew text + re-indexing)