MCP Video Recognition Server

An MCP server that describes images, transcribes audio, and summarizes video from local files. It talks to Google Gemini by default, or to any OpenAI-compatible endpoint such as OpenRouter.

Features

• Pick your provider: Google Gemini (default) or an OpenAI-compatible endpoint
• Three MCP tools for local images, audio, and video
• Optional Gemini model fallback, plus a final OpenAI-compatible backup
• Opt-in multi-perspective parallel inference via PARALLEL_PROMPTS (see the Configuration Reference)

Model support varies by provider. Picking the OpenAI-compatible provider does not mean every endpoint or model handles every media type. And the server never swaps in a different model or provider on its own.

Prerequisites

• Node.js 18.0.0 or later
• An API key for your provider:
  • Gemini: GOOGLE_API_KEY
  • OpenAI-compatible: OPENAI_COMPATIBLE_API_KEY

Install

git clone https://github.com/yourusername/mcp-video-recognition.git
cd mcp-video-recognition
npm install
npm run build

Quickstart

Add the server to your MCP client config and point it at the built dist/index.js:

{
  "mcpServers": {
    "video-recognition": {
      "command": "node",
      "args": ["/path/to/mcp-video-recognition/dist/index.js"],
      "env": {
        "GOOGLE_API_KEY": "your_google_api_key"
      }
    }
  }
}

On Windows, use forward slashes or doubled backslashes (\\) in the path. Save the file and reconnect your MCP client. Environment variables must be present in the MCP client's env configuration; the server process does not automatically inherit values merely because a .env file exists beside it.

For OpenRouter or another OpenAI-compatible endpoint, set RECOGNITION_PROVIDER=openai-compatible and fill in its variables. Configuration has a ready-made example.

Standalone Client

The repository includes a client that builds the project, spawns the server over stdio, performs MCP initialization, and lists the tools. It loads .env from the repository root when that file exists; already-set process environment variables take precedence.

# Connect and list tools
npm run client

# Connect and call a tool
npm run client -- --tool image_recognition --args '{"filepath":"C:/media/example.png"}'

Use --help for server-path, working-directory, env-file, and timeout options:

npm run client -- --help

A tool error exits with status 2 and prints the complete MCP error result. Connection or configuration failures exit with status 1 and include the spawned server's stderr diagnostics.

Other install options (FLUJO)

With FLUJO:

1. Click Add Server.
2. Paste the GitHub URL.
3. Click Parse, Clone, Install, Build and Save.

Configuration

The server reads environment variables. These are the ones you'll touch most:

Variable	Default	Purpose
RECOGNITION_PROVIDER	gemini	gemini or openai-compatible
GOOGLE_API_KEY	none	Gemini API key
GEMINI_MODEL	gemini-3.5-flash	Gemini model to use
OPENAI_COMPATIBLE_API_KEY	none	OpenAI-compatible API key
OPENAI_COMPATIBLE_BASE_URL	none	Endpoint base URL
OPENAI_COMPATIBLE_MODEL	none	Model to use
ALLOWED_MEDIA_ROOTS	none	Media directories for the OpenAI-compatible provider and Gemini backup
TRANSPORT_TYPE	stdio	stdio or streamable-http; sse is a legacy alias
MCP_HOST	127.0.0.1	Bind address for Streamable HTTP
PORT	3000	Bind port for Streamable HTTP
LOG_LEVEL	info	Diagnostic threshold; all logs go to stderr

A bad value stops startup. Nothing gets fixed silently.

The Configuration Reference has the full variable list, validation rules, an OpenRouter example, and the supported media types. For Gemini model fallback and the final backup, read the Provider Recovery Reference.

Tools

You get three MCP tools. Each takes a local filepath, an optional prompt (default Describe this content), and an optional modelname override.

• image_recognition - describe an image
• audio_recognition - transcribe or describe audio
• video_recognition - describe a video

Example:

{
  "name": "video_recognition",
  "arguments": {
    "filepath": "/path/to/video.mp4",
    "prompt": "Describe what happens in this video"
  }
}

Security

• HTTPS is required by default. Plain HTTP only works for a local endpoint you explicitly enable.
• The OpenAI-compatible provider and the Gemini backup read media only from directories listed in ALLOWED_MEDIA_ROOTS. Containment is recursive, so specifying a parent folder (e.g. C:\Projects or ${workspaceFolder}) covers all repositories, subfolders, and media files inside it.
• Keys stay in the process environment. Don't commit real keys.

The Security Reference covers endpoint rules, resource limits, and incident response.

MCP Inspector and Streamable HTTP

For stdio Inspector use, build first and make sure the API key is in Inspector's server environment. In PowerShell:

$env:GOOGLE_API_KEY = "your_google_api_key"
npm run debug

For HTTP, start the server separately:

$env:GOOGLE_API_KEY = "your_google_api_key"
$env:TRANSPORT_TYPE = "streamable-http"
npm start

Then select Streamable HTTP in Inspector and connect to http://127.0.0.1:3000/mcp. Do not select legacy SSE: the backwards-compatible TRANSPORT_TYPE=sse spelling still starts a Streamable HTTP endpoint.

Listing tools only proves the MCP handshake succeeded. It does not contact the recognition provider or read a media file. A later call can still fail because the key, model, network, or filepath is invalid. The path must identify a file visible to the spawned server process; paths from a different container or host will not work.

Development

# Build, connect through the standalone client, and list tools
GOOGLE_API_KEY=your_api_key npm run client

# Build and run the provider foundation tests
npm run verify:provider-foundation

Project Structure

• src/index.ts: entry point and provider construction
• src/server.ts: MCP server and transport
• src/tools/: the three recognition tools
• src/services/: Gemini and OpenAI-compatible providers
• src/types/: shared types
• src/utils/: helpers

License

MIT