Specter Backend - Agentic Legal AI system (based on Elysia)

October 12, 2025 · View on GitHub

front page

Overview

Specter is an advanced (legal focused) AI agent orchestration system built on the Elysia framework. It features intelligent routing, custom agent creation, and dynamic document management for legal-domain specific AI assistance.

Table of Contents

Get Started (Python from Github)

Prerequisites

  • Python 3.12 or higher
  • Weaviate instance (local or cloud)
  • OpenAI API key (or compatible LLM provider)

Installation

To get the latest development version, you can clone the github repo by running

git clone https://github.com/Siddharth-Khattar/specter-backend.git

move to the working directory via

cd specter-backend

Create a virtual environment with Python (version 3.12+)

python3.12 -m venv .venv
source .venv/bin/activate

and then install "Specter-version" of Elysia via pip

pip install -e .

Done! You can now use the the python package

Configure environment variables

Create a .env file in the project root:

# Weaviate Configuration
WCD_URL=your_weaviate_cluster_url
WCD_API_KEY=your_weaviate_api_key

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key

# Model Configuration
BASE_MODEL=gpt-5o-mini
COMPLEX_MODEL=gpt-5

# Logging
LOGGING_LEVEL=INFO

Get started (App)

Run the app via

elysia start

Then navigate to the settings page, add your required API keys, Weaviate cloud cluster details and specify your models.

Start frontend by opening specter-frontend repository and following instructions.

Using the System

Uploading Documents

Documents can be uploaded to create agent-specific knowledge bases.

Form Data:

  • file: The document file (PDF, TXT, DOCX, etc.)
  • user_id: User identifier
  • agent_name: (Optional) Associate document with specific agent
  • collection_name: (Optional) Custom collection name

Document Processing

  1. Documents are automatically chunked for optimal retrieval
  2. Text is vectorized using OpenAI embeddings
  3. Metadata is preserved (filename, upload date, agent association)
  4. Documents are stored in agent-specific Weaviate collections

Running Queries with Agent Routing

When you send a query through the chat interface, the system:

  1. Analyzes the query to determine domain/intent
  2. Routes to appropriate agent using the DomainRouter
  3. Executes agent-specific tools with access to relevant knowledge base
  4. Returns structured response with source citations

Response Stream:

{
  "type": "tree_update",
  "payload": {
    "from_node": "domain_routing",
    "to_node": "legal_advisor",
    "reasoning": "Query requires legal expertise and document analysis"
  }
}

{
  "type": "text",
  "payload": {
    "text": "Based on the service agreement, the key obligations include..."
  }
}

Architecture

Agent System Architecture

User Query
    ↓
DomainRouter (analyzes query intent)
    ↓
┌─────────────────────────────────┐
│  Specialized Agent Selection    │
├─────────────────────────────────┤
│ • Civil-law Agent               │
│ • Immigration-law Agent         │
│ • Work-law Agent                │
│ • General Agent (fallback)      │
└─────────────────────────────────┘
    ↓
Agent-Specific Knowledge Base
    ↓
Tool Execution (Query, Aggregate, etc.)
    ↓
Response with Citations

Decision Tree Flow

The system uses a branching decision tree that's visible in the UI:

  1. Root Node: Domain routing evaluation
  2. Branch Nodes: Agent selection and tool choice
  3. Leaf Nodes: Tool execution (query, aggregate, text response)
  4. Result: Comprehensive response with transparent reasoning

Custom Agent Registry

The CustomAgentRegistry class manages all custom agents:

from elysia.tools.domain.custom_agent_registry import CustomAgentRegistry

# Initialize registry
registry = CustomAgentRegistry(logger=logger)

# Load agents for a user
await registry.load_custom_agents(
    user_id="user123",
    client_manager=client_manager
)

# Get available agents
agent_names = registry.get_agent_names()
agent_descriptions = registry.get_agent_descriptions()

Advanced Configuration

Customizing Agent Behavior

Agents can be fine-tuned through several parameters:

tree.add_tool(
    CustomAgentTool,
    branch_id="domain_routing",
    agent_config={
        "temperature": 0.7,
        "max_tokens": 2000,
        "use_reasoning": True
    }
)

Managing Agent Collections

Each agent can have multiple associated collections:

# Create collection for agent
await create_agent_collection(
    user_id="user123",
    agent_name="legal_advisor",
    collection_name="contracts_2024"
)

# Associate with agent
await update_agent_collections(
    agent_id="agent_uuid",
    collections=["contracts_2024", "legal_precedents"]
)

Routing Configuration

The DomainRouter can be configured to adjust sensitivity:

domain_router = DomainRouter(
    confidence_threshold=0.75,  # Minimum confidence for agent selection
    fallback_to_general=True,   # Use general agent if confidence low
    logger=logger
)

Monitoring & Debugging

Tree Visualization

View the decision tree structure:

tree.view()  # Prints formatted tree structure

Troubleshooting

Common Issues

Agent not being selected:

  • Check agent description matches query domain
  • Verify DomainRouter confidence threshold
  • Review agent system prompt for clarity

Documents not being retrieved:

  • Confirm document upload completed successfully
  • Check collection names match agent configuration
  • Verify Weaviate connection and indexing

Routing loops:

  • Review hidden_environment state (cleared on soft_reset)
  • Check for circular tool dependencies
  • Adjust recursion_limit if needed

Best Practices

  1. Agent Specialization: Create focused agents for specific domains rather than general-purpose agents
  2. System Prompts: Write clear, detailed system prompts that define agent expertise and limitations
  3. Document Organization: Group related documents in dedicated collections per agent
  4. Regular Updates: Keep agent knowledge bases current by periodically updating documents
  5. Monitor Performance: Track agent selection accuracy and adjust confidence thresholds