README.md

---

Knowsee demonstrates how to build a multi-agent conversational AI using Google Agent Development Kit (ADK) for the backend and CopilotKit for the frontend, connected via the AG-UI protocol.

---

Contents

• Features
• Prerequisites
• Quick Start
• Configuration
• Architecture
• Demo
• Try Live Demo
• License

Features

🤖 Agentic Architecture

• Multi-Agent Orchestration — Hierarchical delegation via AgentTool with isolated contexts
• A2A Composition — Agent-to-agent patterns without namespace contamination
• Extended Thinking — Gemini 2.5 Pro with dedicated reasoning budget
• AG-UI Protocol — Bidirectional streaming between frontend and backend
• Generative UI — Tool calls, reasoning, and sources as interactive components
• Data Analyst Agent — Natural language to SQL with BigQuery, query tracking, chart widgets
• SSE Event Bus — Live updates without polling

🔍 Retrieval & Grounding

• Vertex AI RAG Engine — Semantic search with team-scoped corpus access
• Web Search Grounding — Google Search with inline citations
• Multi-Source Synthesis — RAG + web + uploads unified in responses
• Google Drive Picker — Import files directly from Drive with OAuth
• Wide Format Support — png, jpeg, gif, webp, pdf, txt, csv, md, html, docx, doc, xlsx, xls, pptx, ppt, odt, ods, odp, rtf, Google Docs/Sheets/Slides

🏗️ Production-Ready

• Better Auth — Email/password, OTP verification, TOTP 2FA
• Pluggable Identity — Supports Google Groups, Azure AD, custom providers
• Permission-Scoped RAG — Users only access corpora their teams own
• Terraform IaC — Cloud Run, Cloud SQL, Vertex AI, KMS
• SOPS Secrets — Encrypted configuration management
• Scheduled Sync — Automated RAG corpus updates from GDrive/OneDrive

Prerequisites

Tool	Version	Purpose
Node.js	20+	Frontend runtime
Python	3.11+	Backend runtime
uv	Latest	Python package manager
Docker	Latest	Local Postgres
gcloud CLI	Latest	GCP authentication

For deployment only: | Terraform | 1.5+ | Infrastructure as Code | | SOPS | Latest | Secrets encryption |

Quick Start

1. Clone and Install

git clone https://github.com/saahil-mehta/knowsee-public.git
cd knowsee-public
make install

2. Configure Environment

cp sagent/.env.example sagent/.env.development
cp web/.env.example web/.env.development

Edit both files with your values:

Variable	Where	Notes
GOOGLE_CLOUD_PROJECT	sagent	GCP Free Tier includes Vertex AI credits
NEXT_PUBLIC_COPILOTKIT_PUBLIC_KEY	web	Free for personal use
BETTER_AUTH_SECRET	web	Generate: openssl rand -base64 32
MAILGUN_*	web	Free tier — required for email OTP

3. Authenticate with GCP

Required for Vertex AI (Gemini models, RAG engine):

make gcp-login

4. Start Development

Ensure Docker Desktop is running, then:

make dev

This automatically:

• Starts local Postgres via Docker
• Runs database migrations
• Launches ADK backend → http://localhost:8000
• Launches Next.js frontend → http://localhost:3000

5. Create an Account

Open http://localhost:3000 and sign up. You'll receive an OTP via email (or check Mailgun logs in sandbox mode).

Configuration

Backend (sagent/.env.development)

Variable	Required	Description
GOOGLE_CLOUD_PROJECT	Yes	GCP project ID for Vertex AI
GOOGLE_CLOUD_LOCATION	Yes	GCP region (default: europe-west1)
GOOGLE_GENAI_USE_VERTEXAI	Yes	TRUE for Vertex AI (recommended) or use API key

Variable	Required	Description
DATABASE_URL	Yes	Postgres connection string
ENVIRONMENT	No	development (default) or production
RAG_SIMILARITY_TOP_K	No	RAG retrieval count (default: 10)
TEAM_MEMBERSHIP_PROVIDER	No	better_auth, google_groups, or azure_ad

Frontend (web/.env.development)

Variable	Required	Description
AGENT_URL	Yes	Backend URL (default: http://localhost:8000)
NEXT_PUBLIC_COPILOTKIT_PUBLIC_KEY	Yes	CopilotKit public key
BETTER_AUTH_URL	Yes	Auth callback URL (default: http://localhost:3000)
BETTER_AUTH_SECRET	Yes	Session encryption key
DATABASE_URL	Yes	Postgres connection string
MAILGUN_API_KEY	No	Mailgun API key (optional)
MAILGUN_DOMAIN	No	Mailgun sending domain
MAILGUN_FROM	No	From address for emails

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              User Browser                                    │
└─────────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  Frontend (Next.js + CopilotKit)                                            │
│  ├── Chat UI with streaming responses                                       │
│  ├── Better Auth (sessions, teams)                                          │
│  └── /api/copilotkit → AG-UI bridge                                         │
└─────────────────────────────────────────────────────────────────────────────┘
                                      │ AG-UI Protocol
                                      ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  Backend (FastAPI + ADK)                                                    │
│  ├── Root Agent (Gemini 2.5 Pro + extended thinking)                        │
│  │   ├── Team Knowledge Agent → Vertex AI RAG                               │
│  │   ├── Web Search Agent → Google Search                                   │
│  │   ├── Data Analyst Agent → BigQuery SQL execution                        │
│  │   └── File Tools → Artifact storage                                      │
│  └── Callbacks: user context, artifacts, widgets                            │
└─────────────────────────────────────────────────────────────────────────────┘
                                      │
                    ┌────────────┬────┴────┬────────────┐
                    ▼            ▼         ▼            ▼
              ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
              │ Postgres │ │ Vertex   │ │ BigQuery │ │   GCS    │
              │ Sessions │ │ AI RAG   │ │ Analytics│ │ Artifacts│
              │ Auth     │ │ Engine   │ │          │ │          │
              └──────────┘ └──────────┘ └──────────┘ └──────────┘

Data Flow

1. User sends message → CopilotKit streams to /api/copilotkit with x-user-id header
2. AG-UI bridge → Forwards to ADK backend via HttpAgent
3. Context injection → User's teams and accessible RAG corpora loaded into state
4. Agent execution → Root agent delegates to sub-agents (RAG, web search, files)
5. Response synthesis → Gemini combines sources with inline citations
6. Streaming response → Flows back through AG-UI to CopilotKit UI

Project Structure

sagent/                          # Backend (ADK + FastAPI)
├── agents/
│   ├── root.py                 # Main orchestrator
│   ├── search.py               # Web search sub-agent
│   ├── data_analyst/           # BigQuery SQL agent
│   └── rag/agent.py            # Team knowledge sub-agent
├── callbacks/                  # Before/after LLM hooks
├── services/rag/               # RAG sync and config
└── main.py                     # FastAPI server

web/                             # Frontend (Next.js + CopilotKit)
├── src/app/
│   ├── api/copilotkit/         # AG-UI bridge
│   ├── api/sessions/           # Session management
│   └── chat/                   # Chat pages
└── src/components/
    ├── charts/                 # Recharts visualisations
    ├── chat/tool-call/         # Modular tool renderers
    └── copilotkit-provider.tsx # Auth header injection

Demo

1. Create an Account

Sign up with email and password.

2. Verify Your Email

An OTP is sent to your inbox.

---

3. Welcome to Knowsee

After verification, you're in.

4. Chat with Personality

Ask anything — get up-to-date answers with a touch of humour.

5. Light Mode

For the light-mode lovers.

6. Sign In & Out

Seamless authentication flow.

7. Google Drive Import

Import files directly from your Google Drive into conversations.

Demo coming soon

Try Live Demo

→ Try Knowsee Live

License

Apache 2.0