DeepImageSearch Web Demo

March 16, 2026 · View on GitHub

A FastAPI + React web demo that lets you browse a photo library, search photos with natural language, and watch the AI Agent's reasoning process in real time.

How It Differs from the ImageSeeker CLI

You may already be familiar with ImageSeeker/src/demo.py:

# ImageSeeker CLI (command-line args, full environment required)
python -m src.demo \
  --dataset-path ../DISBench/ \
  --retriever Qwen/Qwen3-VL-Embedding-8B \
  --agent openai/gpt-4o \
  --user-id "15352839@N00" \
  --query "Find all photos of sunsets"

Key differences:

	ImageSeeker CLI (`src/demo.py`)	Web Demo (`demo/server.py`)
Launch	`python -m src.demo --args...`	`uvicorn demo.server:app`
Configuration	CLI flags (`--agent`, `--retriever`)	Environment variables (`DEMO_AGENT`, `DEMO_RETRIEVER`)
Data source	Requires DISBench dataset (`--dataset-path`)	Any image folder (`DEMO_IMAGE_DIR`)
User selection	Must specify `--user-id`	Loads the entire image directory automatically
Querying	One-shot `--query "..."`	Interactive browser input, unlimited searches
Output	Terminal text	Browser UI (animations, photo lightbox, reasoning steps)
Mock mode	None; requires Retriever + LLM	Yes; a single command shows the full UI
Prerequisites	vLLM + FAISS index + LLM API key	Mock mode: none

The key takeaway: the CLI requires completing all preparation steps in the ImageSeeker README (building FAISS indices, deploying the vLLM Retriever, configuring API keys), while the Web Demo's Mock mode runs with nothing but an image folder.

Quick Start (3 Steps)

Step 1: Install Dependencies

# Backend
cd demo
pip install -r requirements.txt

# Frontend build (one-time)
cd frontend
npm install && npm run build
cd ../..

Step 2: Configure `.env`

Create a .env file in the demo/ directory (a template is provided):

# === Web Demo Configuration ===
DEMO_IMAGE_DIR=DISBench/images/15352839@N00   # Image directory (required)
DEMO_MOCK=1                                    # Mock mode (no external services needed)

# === Only needed for Live mode (skip for Mock) ===
# DEMO_METADATA_PATH=DISBench/metadata/15352839@N00.jsonl
# DEMO_RETRIEVER=Qwen/Qwen3-VL-Embedding-8B
# DEMO_AGENT=openai/gpt-4o

# === ImageSeeker configuration (Live mode) ===
# OPENAI_API_KEY=sk-...
# SEARCH_API_URL_MAP={"Qwen/Qwen3-VL-Embedding-8B": "http://localhost:51282/global_user_search"}

The .env file lives at demo/.env and is loaded automatically regardless of your working directory.

You can also point to any image folder instead of DISBench:

DEMO_IMAGE_DIR=my_photos    # Supports jpg/jpeg/png/webp/bmp/gif

Step 3: Launch

# Run from the project root (auto-reads .env)
uvicorn demo.server:app --port 8000

Open http://localhost:8000 in your browser to see:

A photo library grouped by date
A search bar at the top for AI-powered search
Click a suggestion or type any query to watch the Agent's animated reasoning process

Note: In Mock mode, search results are random and do not reflect your query. Its purpose is to showcase the UI interactions.

Live Mode (Real AI Search)

Live mode runs the actual ImageSeeker Agent, so the prerequisites are the same as the CLI:

Prerequisites (Same as ImageSeeker)

FAISS index built — follow "Build the retriever index" in ImageSeeker/README.md
Retriever service deployed — bash scripts/deploy_vllm_retriever.sh ... running with "Application startup complete."
.env configured — with OPENAI_API_KEY (or other LLM key) and SEARCH_API_URL_MAP
ImageSeeker dependencies installed — cd ImageSeeker && pip install -r requirements.txt

Launch Live Mode

Update your demo/.env (set DEMO_MOCK=0, add Retriever and metadata paths):

# demo/.env
DEMO_IMAGE_DIR=DISBench/images/15352839@N00
DEMO_METADATA_PATH=DISBench/metadata/15352839@N00.jsonl
DEMO_MOCK=0
DEMO_RETRIEVER=Qwen/Qwen3-VL-Embedding-8B
DEMO_AGENT=openai/gpt-4o

# ImageSeeker configuration
OPENAI_API_KEY=sk-...
SEARCH_API_URL_MAP={"Qwen/Qwen3-VL-Embedding-8B": "http://localhost:51282/global_user_search"}

Then launch (auto-reads .env):

uvicorn demo.server:app --port 8000

Using Your Own Photos (Non-DISBench)

To use your own photos instead of the DISBench dataset:

Place photos in a folder, e.g., my_photos/
Create a metadata file my_photos_metadata.jsonl with one JSON object per line:

{"photo_id": "IMG_001", "taken_time": "2024-06-15 14:30:00", "address": "United States, California, San Francisco"}
{"photo_id": "IMG_002", "taken_time": "2024-07-20 09:15:00", "address": "Japan, Tokyo, Shibuya"}

Where photo_id matches the filename (without extension), and taken_time/address are used by the Agent's metadata tools.

Build a FAISS index and deploy the Retriever for your photos (the most complex step; see the ImageSeeker README)
Configure .env accordingly and run uvicorn demo.server:app --port 8000

Environment Variables Reference

Variable	Required	Default	Description
`DEMO_IMAGE_DIR`	Yes	-	Image directory path (e.g., `DISBench/images/15352839@N00`)
`DEMO_MOCK`	No	`false`	Set to `1` to enable Mock mode (no external services needed)
`DEMO_METADATA_PATH`	Live mode	-	Metadata JSONL file path
`DEMO_RETRIEVER`	Live mode	-	Retriever model name (e.g., `Qwen/Qwen3-VL-Embedding-8B`)
`DEMO_AGENT`	No	`openai/gpt-4o`	Agent LLM model
`DEMO_MEMORYMANAGER`	No	`openai/gpt-4o-mini`	Memory compression model
`DEMO_MAX_TURNS`	No	`30`	Maximum Agent iteration turns
`DEMO_MAX_CONTEXT_TOKENS`	No	`65536`	Context token budget
`DEMO_WITHOUT`	No	`""`	Comma-separated list of tools to exclude
`DEMO_USER_ID`	No	basename of `DEMO_IMAGE_DIR`	Override the user_id inferred from `DEMO_IMAGE_DIR`

CLI parameter mapping:

CLI Flag	Environment Variable	Notes
`--dataset-path`	N/A	Web Demo splits this into `DEMO_IMAGE_DIR` + `DEMO_METADATA_PATH`
`--user-id`	N/A	Web Demo loads the entire directory; no user-id needed
`--retriever`	`DEMO_RETRIEVER`	Same meaning
`--agent`	`DEMO_AGENT`	Same meaning
`--query`	N/A	Web Demo accepts queries in the browser
`--max-turns`	`DEMO_MAX_TURNS`	Same meaning
`--mock`	`DEMO_MOCK`	Web Demo only; no CLI equivalent

Frontend Development Mode

To modify the frontend with hot-reload:

# Terminal 1: Start the backend (serves API)
DEMO_IMAGE_DIR=my_photos DEMO_MOCK=1 uvicorn demo.server:app --port 8000

# Terminal 2: Start the frontend dev server (Vite with HMR)
cd demo/frontend
npm run dev
# Visit http://localhost:5173 (auto-proxies /api requests to port 8000)

Directory Structure

demo/
├── server.py                 # FastAPI backend (SSE search stream, image serving, library pagination API)
├── mock.py                   # Mock Agent (simulates a 3-turn reasoning flow, no external services)
├── requirements.txt          # Python deps: fastapi, uvicorn, sse-starlette, langchain-core, langchain-openai, langgraph
├── __init__.py
└── frontend/                 # React frontend
    ├── index.html
    ├── package.json          # React 19, Zustand 5, Framer Motion 11, Tailwind CSS 4
    ├── vite.config.ts        # Dev proxy: /api → localhost:8000
    └── src/
        ├── main.tsx          # Entry point
        ├── App.tsx           # Root component
        ├── index.css         # Design tokens + light/dark themes + animations
        ├── api/
        │   ├── types.ts      # SSEEvent, Step, DateGroup, etc.
        │   └── sse.ts        # SSE client
        ├── stores/
        │   ├── galleryStore.ts   # Gallery state (groups, pagination, search toggle)
        │   └── searchStore.ts    # Search state (steps, results, theme)
        ├── lib/
        │   ├── animations.ts     # Framer Motion animation variants
        │   └── cn.ts
        └── components/
            ├── NavBar.tsx            # Navigation bar (branding + AI search entry)
            ├── ThemeToggle.tsx       # Light/dark theme toggle
            ├── SearchOverlay.tsx     # Fullscreen search overlay
            ├── SuggestionPills.tsx   # Search suggestions (8 preset queries)
            ├── SearchView.tsx        # Search results (left: steps, right: photos)
            ├── StepTimeline.tsx      # Agent step timeline
            ├── StepItem.tsx          # Single step (purple=reasoning, cyan=tool, green=photos)
            ├── ResultsHero.tsx       # Search results display
            ├── GalleryView.tsx       # Photo gallery (infinite scroll)
            ├── DateSection.tsx       # Date group (adaptive grid)
            ├── DateHeader.tsx        # Date header bar (date + location)
            ├── GalleryTile.tsx       # Gallery photo thumbnail
            ├── GridDensityControl.tsx # Grid density control
            ├── PhotoCanvas.tsx       # Photo canvas renderer
            ├── PhotoCard.tsx         # Search result photo card (with match score)
            ├── PhotoGrid.tsx         # Photo grid layout
            └── Lightbox.tsx          # Fullscreen photo lightbox

Design Notes

UI Design

Tech-forward theme: indigo/purple/cyan gradients, frosted-glass nav bar, dark mode by default
AI search prominence: large centered search entry in the nav bar with a gradient "AI" badge
Reasoning visualization: each Agent step rendered in a three-color scheme (purple = reasoning, cyan = tool call, green = photos found); photos display match scores and are clickable for full-size viewing
Responsive: adapts to mobile, tablet, desktop, and ultrawide screens
Light/dark toggle: global theme switch via CSS custom properties

Backend Architecture

SSE streaming: each reasoning step (reasoning → tool_call → tool_result → photos → result) is pushed to the frontend in real time
Library pagination: /api/library returns date-grouped photos in reverse chronological order with cursor pagination; the frontend uses IntersectionObserver for infinite scrolling
Image serving: /api/photos/{id} serves local image files directly with cache headers
Live mode: calls ImageSeeker/src/agent/graph.py's build_graph() + graph.astream() directly, sharing the exact same Agent core as the CLI