🤖 Agentic RAG AI Backend

December 10, 2025 · View on GitHub

A production-ready FastAPI backend demonstrating Agentic RAG - combining Retrieval-Augmented Generation with autonomous tool-calling capabilities.

🎬 This is an extension of yt-rag - If you're new here, check out the original RAG tutorial first!

📹 YouTube Tutorial: YouTube Link

☕️ Support: Buy me a coffee

🤖 Discord: Join our community

🆕 What's New in Agentic RAG?

Feature	yt-rag (v1)	yt-agentic-rag (v2)
Vector Search	✅	✅
RAG Q&A	✅	✅
Tool Calling	❌	✅
Agent Reasoning Loop	❌	✅
Calendar Scheduling	❌	✅
Email Sending	❌	✅
Multi-turn Chat History	❌	✅
Multi-step Actions	❌	✅

The Key Difference

Traditional RAG: Query → Retrieve → Answer

Agentic RAG: Query → Retrieve → Reason → Decide → Act → Answer

The agent can now:

Retrieve relevant context from your knowledge base
Reason about whether the context is relevant to the action
Decide if tools need to be called
Act by executing tools (schedule meetings, send emails)
Answer with citations and confirmation of actions taken

📁 Project Structure

yt-agentic-rag/
│
├── 📂 app/                              # 🚀 PRODUCTION APPLICATION
│   │
│   ├── 📂 agents/                       # 🤖 AI AGENTS (the star of the show!)
│   │   ├── __init__.py                  # Exports agent_service
│   │   ├── orchestrator.py              # Main agent reasoning loop
│   │   └── 📂 tools/                    # Agent capabilities
│   │       ├── __init__.py              # Exports tool_registry
│   │       ├── base.py                  # Abstract base class for tools
│   │       ├── registry.py              # Tool registration & execution
│   │       ├── calendar_tool.py         # Google Calendar integration
│   │       └── email_tool.py            # Gmail integration
│   │
│   ├── 📂 services/                     # 📦 Core business logic
│   │   ├── __init__.py
│   │   ├── rag.py                       # RAG pipeline (retrieve → augment → generate)
│   │   ├── embedding.py                 # Vector embedding service
│   │   ├── chat.py                      # LLM chat completion service
│   │   └── chunker.py                   # Text chunking utilities
│   │
│   ├── 📂 schemas/                      # 📋 API request/response definitions
│   │   ├── __init__.py
│   │   ├── requests.py                  # Input validation (AgentRequest, etc.)
│   │   ├── responses.py                 # Output formats (AgentResponse, etc.)
│   │   ├── entities.py                  # Database entity models
│   │   └── tool_schemas.py              # LLM function-calling definitions
│   │
│   ├── 📂 config/                       # ⚙️ Configuration & infrastructure
│   │   ├── __init__.py
│   │   ├── settings.py                  # Environment variable management
│   │   └── database.py                  # Supabase connection & operations
│   │
│   ├── 📂 data/                         # 📚 Static data
│   │   ├── __init__.py
│   │   └── default_documents.py         # Sample documents for RAG
│   │
│   └── main.py                          # FastAPI app & route definitions
│
├── 📂 devtools/                         # 🛠️ Development & debugging utilities
│   ├── README.md                        # How to use these tools
│   ├── test_setup.py                    # Verify your setup works
│   ├── verify_rag.py                    # Prove RAG is working
│   ├── check_dimensions.py              # Debug embedding dimensions
│   └── chatbot.py                       # Terminal chat client
│
├── 📂 credentials/                      # 🔐 Google service account (gitignored)
│   └── service_account.json             # Your GCP service account key
│
├── 📂 sql/                              # 🗄️ Database setup
│   └── init_supabase.sql                # Supabase schema & pgvector setup
│
├── 📂 static/                           # 🎨 Frontend assets
│   └── chat.html                        # Web chat interface
│
├── main.py                              # Root entry point (re-exports app/main.py)
├── requirements.txt                     # Python dependencies
├── Dockerfile                           # Container configuration
├── .env.example                         # Environment variable template
├── DEPLOYMENT.md                        # Cloud Run deployment guide
└── README.md                            # This file

🚀 Complete Setup Guide (From Scratch)

Prerequisites

Python 3.11+ - Download
Git - Download
Supabase Account - Sign up free
OpenAI API Key - Get key
Google Cloud Account (for calendar/email tools) - Console

Step 1: Clone and Install

# Clone the repository
git clone https://github.com/ShenSeanChen/yt-agentic-rag.git
cd yt-agentic-rag

# Create virtual environment
python3.11 -m venv venv

# Activate virtual environment
source venv/bin/activate  # macOS/Linux
# OR
venv\Scripts\activate     # Windows

# Install dependencies
pip install -r requirements.txt

Step 2: Set Up Supabase (Vector Database)

2.1 Create a Supabase Project

Go to supabase.com and sign in
Click "New Project"
Choose a name (e.g., agentic-rag)
Set a secure database password (save this!)
Select a region close to you
Click "Create new project" and wait for setup

2.2 Get Your API Keys

In your project, go to Settings → API
Copy these values:
- Project URL (e.g., https://abc123.supabase.co)
- anon public key (starts with eyJ...)
- service_role key (starts with eyJ...) - ⚠️ Keep this secret!

2.3 Initialize the Database Schema

In Supabase, go to SQL Editor
Click "New Query"
Copy the contents of sql/init_supabase.sql and paste it
Click "Run" to execute

This creates:

rag_chunks table for storing document embeddings
match_chunks function for vector similarity search
Required indexes for performance

Step 3: Set Up OpenAI

Go to platform.openai.com/api-keys
Click "Create new secret key"
Copy the key (starts with sk-...)

Step 4: Set Up Google Cloud (For Calendar & Email Tools)

⚠️ Skip this step if you only want RAG without tool calling.

4.1 Create a Google Cloud Project

Go to console.cloud.google.com
Click the project dropdown → "New Project"
Name it (e.g., agentic-rag-tools)
Click "Create"

4.2 Enable APIs

Go to APIs & Services → Library
Search and enable:
- Google Calendar API
- Gmail API

4.3 Create a Service Account

Go to IAM & Admin → Service Accounts
Click "Create Service Account"
Name: agentic-rag-service
Click "Create and Continue"
Skip the optional steps, click "Done"

4.4 Generate a Key

Click on your new service account
Go to Keys tab
Click "Add Key" → "Create new key"
Choose JSON
Click "Create" - a file downloads
Move this file to credentials/service_account.json in your project

4.5 Enable Domain-Wide Delegation (Google Workspace)

Required if using a Google Workspace account (e.g., @yourcompany.com)

In the service account details, click "Show Advanced Settings"
Copy the Client ID (a long number)
Go to admin.google.com (Google Workspace Admin)
Navigate to Security → API Controls → Domain-wide Delegation
Click "Add new"
Paste the Client ID

Add these OAuth scopes:

https://www.googleapis.com/auth/calendar
https://www.googleapis.com/auth/gmail.send

Click "Authorize"

Step 5: Configure Environment Variables

# Copy the template
cp .env.example .env

# Edit with your values
nano .env  # or use your preferred editor

Fill in your .env file:

# ===========================================
# SUPABASE CONFIGURATION (Required)
# ===========================================
SUPABASE_URL=https://your-project-id.supabase.co
SUPABASE_ANON_KEY=your-supabase-anon-key
SUPABASE_SERVICE_ROLE_KEY=your-supabase-service-role-key

# ===========================================
# OPENAI CONFIGURATION (Required)
# ===========================================
OPENAI_API_KEY=sk-your-openai-api-key
OPENAI_EMBED_MODEL=text-embedding-3-small
OPENAI_CHAT_MODEL=gpt-4o

# ===========================================
# AI PROVIDER SETTINGS
# ===========================================
AI_PROVIDER=openai

# ===========================================
# GOOGLE API CONFIGURATION (For Tools - Optional)
# ===========================================
GOOGLE_SERVICE_ACCOUNT_PATH=credentials/service_account.json
GOOGLE_CALENDAR_EMAIL=your-email@yourcompany.com
GOOGLE_CALENDAR_ID=primary

# ===========================================
# APPLICATION SETTINGS
# ===========================================
ENVIRONMENT=development
LOG_LEVEL=INFO

Step 6: Verify Setup

# Run the setup verification script
python devtools/test_setup.py

You should see:

✅ All modules imported successfully
✅ Environment variables configured
✅ Database connection successful
✅ Database schema validated
✅ Successfully seeded X document chunks
✅ RAG query successful!
🎉 ALL TESTS PASSED!

Step 7: Start the Server

uvicorn main:app --reload --port 8000

Visit:

API Docs: http://localhost:8000/docs
Chat UI: http://localhost:8000/chat
Health Check: http://localhost:8000/healthz

📚 API Endpoints

Health & Info

Endpoint	Method	Description
`/healthz`	GET	Health check with database status
`/tools`	GET	List available agent tools
`/documents`	GET	List documents in knowledge base

RAG (Question Answering)

# Traditional RAG - Answer questions from knowledge base
curl -X POST http://localhost:8000/answer \
  -H "Content-Type: application/json" \
  -d '{"query": "What is your return policy?", "top_k": 6}'

🆕 Agent (RAG + Tool Calling)

# Agentic RAG - Can answer questions AND take actions
curl -X POST http://localhost:8000/agent \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Schedule a consultation call with john@example.com for tomorrow at 2pm",
    "top_k": 6
  }'

🆕 Multi-turn Conversations (Chat History)

# Include chat history for context-aware conversations
curl -X POST http://localhost:8000/agent \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Make it 30 minutes",
    "chat_history": [
      {"role": "user", "content": "Schedule a call with john@example.com tomorrow at 2pm"},
      {"role": "assistant", "content": "I can schedule that. How long should the meeting be?"}
    ],
    "top_k": 6
  }'

🔧 Adding New Tools

The architecture makes it easy to add new agent capabilities:

1. Create the Tool (`app/agents/tools/my_tool.py`)

# Directory: yt-agentic-rag/app/agents/tools/my_tool.py

from typing import Dict, Any
from .base import BaseTool

class MyTool(BaseTool):
    """Description of what this tool does."""
    
    @property
    def name(self) -> str:
        return "my_tool_name"
    
    @property
    def description(self) -> str:
        return "A clear description for the LLM"
    
    async def execute(self, param1: str, param2: int = 10, **kwargs) -> Dict[str, Any]:
        """Execute the tool with given parameters."""
        # Your implementation here
        result = do_something(param1, param2)
        return self._success_response({"result": result})

# Export singleton instance
my_tool = MyTool()

2. Register in Registry (`app/agents/tools/registry.py`)

from .my_tool import my_tool

class ToolRegistry:
    def _register_default_tools(self):
        self.register(calendar_tool)
        self.register(email_tool)
        self.register(my_tool)  # Add your tool here

3. Add Tool Schema (`app/schemas/tool_schemas.py`)

TOOL_DEFINITIONS.append({
    "type": "function",
    "function": {
        "name": "my_tool_name",
        "description": "A clear description for the LLM to understand when to use this tool",
        "parameters": {
            "type": "object",
            "properties": {
                "param1": {
                    "type": "string",
                    "description": "What this parameter is for"
                },
                "param2": {
                    "type": "integer",
                    "description": "Optional parameter with default"
                }
            },
            "required": ["param1"]
        }
    }
})

🎬 Demo Scenarios

Scenario 1: RAG Influences Tool Parameters

User: "Schedule a standard consultation meeting with Emma"

Agent:

Retrieves RAG context → finds "Standard consultation calls are 30 minutes"
Uses this info to set duration = 30 minutes
Calls calendar tool with correct duration
Responds with citation [scheduling_consultation_v1]

Scenario 2: Multi-turn Conversation

User: "I want to schedule a meeting" Agent: "I'd be happy to help! Who should I invite and when?" User: "With john@example.com tomorrow at 3pm" Agent: "What type of meeting? (consultation, demo, support call)" User: "A product demo" Agent: Creates 45-minute meeting based on RAG context

Scenario 3: Pure Tool Call (RAG Irrelevant)

User: "Send an email to john@example.com saying hello"

Agent:

Retrieves context → policies not relevant
Ignores irrelevant context
Calls email tool directly
Confirms action

🐳 Docker Deployment

# Build the image
docker build -t yt-agentic-rag .

# Run with environment variables
docker run -p 8080:8080 --env-file .env yt-agentic-rag

☁️ Google Cloud Run Deployment

1. Deploy the Container

Deploy your Docker image to Cloud Run via the console or CLI.

2. Set Up Service Account Secret

The Google Calendar/Gmail tools require a service account JSON file. Since you can't commit credentials to git, use Secret Manager:

# Create the secret from your local service account file
gcloud secrets create service_account \
  --data-file=credentials/service_account.json \
  --project=YOUR_PROJECT_ID

# Grant Cloud Run access to the secret
gcloud secrets add-iam-policy-binding service_account \
  --member="serviceAccount:YOUR_PROJECT_NUMBER-compute@developer.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor" \
  --project=YOUR_PROJECT_ID

💡 Find your project number in Cloud Console → IAM & Admin → Settings

3. Mount Secret as Volume in Cloud Run

Important: The service account JSON must be mounted as a file, not an environment variable.

Step A: Create Volume (Volumes Tab)

Go to Cloud Run → Your Service → Edit & Deploy New Revision
Go to Volumes tab → Add Volume
Configure:
- Volume type: Secret
- Volume name: secret-1
- Secret: service_account
- Path 1: service_account.json (just the filename, not full path!)
- Version: latest
Click Done