Zero-Vector: Production-Ready Vector Database Server

June 15, 2025 · View on GitHub

A high-performance, standalone vector database server built with Node.js, optimized for AI embedding applications with comprehensive monitoring and security features.

🎯 Project Overview

Zero-Vector is a complete implementation of the technical specification outlined in DEVTEAM-HANDOFF.md, providing:

Memory-Efficient Vector Storage: 2GB optimized storage supporting 349,525+ vectors
High-Performance Similarity Search: Cosine, Euclidean, and dot product metrics with sub-50ms query times
Production-Ready Architecture: Three-tier design with comprehensive error handling and monitoring
RESTful API: Complete CRUD operations for vectors with batch processing capabilities
SQLite Integration: Persistent metadata storage with full-text search capabilities
Security Middleware: Helmet.js, CORS, rate limiting, and input validation
Real-time Monitoring: Structured logging, performance metrics, and health checks

🚀 Quick Start

Prerequisites

Node.js 18+
npm or yarn
2GB+ available RAM (recommended)

Installation

# Clone the repository
git clone <repository-url>
cd zero-vector

# Install server dependencies
cd server
npm install

# Setup database
npm run setup:database

# Generate API key for MCP server (if needed)
npm run generate:api-key

# Start the server
npm start

The server will start on http://localhost:3000 with the following endpoints available:

GET /health - Health check and system status
POST /api/vectors - Insert vectors
POST /api/vectors/search - Similarity search
GET /api/vectors/:id - Retrieve specific vector
PUT /api/vectors/:id - Update vector
DELETE /api/vectors/:id - Delete vector

Quick Test

# Test the API with the included test script
node test-vector-api.js

Usage

Basic Vector Operations

1. Insert a Vector

curl -X POST http://localhost:3000/api/vectors \
  -H "Content-Type: application/json" \
  -d '{
    "vector": [0.1, 0.2, 0.3, /* ... 1536 dimensions */],
    "metadata": {
      "content": "Sample text",
      "source": "user_input",
      "tags": ["example"]
    }
  }'

2. Search Similar Vectors

curl -X POST http://localhost:3000/api/vectors/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": [0.1, 0.2, 0.3, /* ... 1536 dimensions */],
    "limit": 10,
    "threshold": 0.7
  }'

3. Get Vector by ID

curl http://localhost:3000/api/vectors/vector_123

4. Check System Health

curl http://localhost:3000/health

API Key Management

The Zero-Vector server includes a secure CLI-based API key generator for MCP server authentication:

Interactive API Key Generation:

cd server
npm run generate:api-key

Quick MCP Key Generation:

cd server
npm run generate:mcp-key

Command Line Options:

node scripts/generate-api-key.js --name "Production Key" --permissions "read,write,vectors:read,vectors:write" --rate-limit 5000 --expires-in-days 180

Available Permissions:

read - Read access to all endpoints
write - Write access for creating/updating data
vectors:read - Vector-specific read operations
vectors:write - Vector-specific write operations
personas:read - Persona-specific read operations
personas:write - Persona-specific write operations
admin - Full administrative access

Using Generated API Keys:

# Set environment variable for MCP server
export ZERO_VECTOR_API_KEY="vdb_your_generated_key_here"

# Or use in HTTP headers
curl -H "X-API-Key: vdb_your_generated_key_here" http://localhost:3000/api/vectors

Integration Examples

Node.js Client

const axios = require('axios');

const client = axios.create({
  baseURL: 'http://localhost:3000/api'
});

// Insert vector
await client.post('/vectors', {
  vector: new Array(1536).fill(0).map(() => Math.random()),
  metadata: { content: 'Example text' }
});

// Search vectors
const results = await client.post('/vectors/search', {
  query: new Array(1536).fill(0).map(() => Math.random()),
  limit: 5
});

Python Client

import requests
import numpy as np

base_url = 'http://localhost:3000/api'

# Insert vector
response = requests.post(f'{base_url}/vectors', json={
    'vector': np.random.rand(1536).tolist(),
    'metadata': {'content': 'Example text'}
})

# Search vectors
response = requests.post(f'{base_url}/vectors/search', json={
    'query': np.random.rand(1536).tolist(),
    'limit': 5
})

📊 Performance Characteristics

Based on current implementation and testing:

Vector Storage: ~6MB per 1000 vectors (1536 dimensions)
Search Performance: <50ms for 10,000+ vector corpus
Memory Efficiency: 99.9% utilization of allocated buffer space
Throughput: 1000+ vectors/second insertion rate
Capacity: 349,525 vectors in 2GB configuration (configurable)

🏗️ Architecture

System Components

┌─────────────────────────────────────────────────────────────┐
│                    Zero-Vector Server                       │
├─────────────────────────────────────────────────────────────┤
│  Controller Layer    │  Express Routes & Middleware         │
│  Service Layer       │  Vector Operations & Business Logic  │
│  Repository Layer    │  SQLite Storage & Vector Store       │
└─────────────────────────────────────────────────────────────┘

Core Technologies

Backend: Node.js + Express.js
Vector Storage: Float32Array buffers with magnitude caching
Database: SQLite for metadata persistence
Security: Helmet.js, CORS, input validation
Monitoring: Winston logging, performance middleware
Testing: Custom API test suite

📖 API Documentation

Vector Operations

Insert Vector

POST /api/vectors
Content-Type: application/json

{
  "vector": [0.1, 0.2, 0.3, ...], // 1536 dimensions
  "metadata": {
    "content": "Sample text",
    "source": "user_input",
    "tags": ["example"]
  }
}

Search Vectors

POST /api/vectors/search
Content-Type: application/json

{
  "query": [0.1, 0.2, 0.3, ...], // 1536 dimensions
  "limit": 10,
  "threshold": 0.7,
  "metric": "cosine",
  "include_metadata": true,
  "filters": {
    "source": "user_input"
  }
}

Health Monitoring

GET /health - Basic health status
GET /health/detailed - Comprehensive system metrics
GET /health/ready - Kubernetes readiness probe
GET /health/live - Kubernetes liveness probe

🔧 Configuration

Environment Variables

# Server Configuration
NODE_ENV=development
PORT=3000
HOST=localhost

# Vector Database Settings
MAX_MEMORY_MB=2048        # Vector store memory allocation
DEFAULT_DIMENSIONS=1536   # Default vector dimensions
INDEX_TYPE=hnsw          # Future: HNSW indexing
DISTANCE_METRIC=cosine   # Similarity metric

# Database
DB_PATH=./data/vectordb.sqlite

# Security
JWT_SECRET=your-secret-key
API_KEY_SALT_ROUNDS=12

# Monitoring
LOG_LEVEL=info
METRICS_ENABLED=true

📁 Project Structure

zero-vector/
├── server/                    # Node.js backend server
│   ├── src/
│   │   ├── config/           # Configuration management
│   │   ├── middleware/       # Express middleware
│   │   ├── repositories/     # Data access layer
│   │   ├── routes/          # API route definitions
│   │   ├── services/        # Business logic layer
│   │   └── utils/           # Helper utilities
│   ├── scripts/             # Database setup scripts
│   ├── data/               # SQLite database files
│   ├── logs/               # Application logs
│   └── README.md           # Server documentation
├── test-vector-api.js       # API testing script
├── DEVTEAM-HANDOFF.md      # Original technical specification
└── README.md               # This file

🛡️ Security Features

Input Validation: Comprehensive request validation with detailed error messages
Security Headers: Helmet.js implementation with CSP policies
CORS Protection: Configurable cross-origin resource sharing
Error Handling: Structured error responses without sensitive data exposure
Logging: Comprehensive audit logging for all operations

📈 Monitoring & Observability

Structured Logging

Format: JSON with Winston
Levels: error, warn, info, debug
Context: Request IDs, user info, performance metrics

Performance Metrics

Memory utilization and vector store statistics
Request/response times and throughput
Cache hit rates and similarity computation performance
System resources (CPU, memory, uptime)

Health Checks

Basic: Service availability and database connectivity
Detailed: Comprehensive system metrics and performance data
Kubernetes: Ready/live probes for container orchestration

🚀 Deployment

Development

cd server
npm run dev  # Start with nodemon for auto-restart

Production

cd server
npm start

Docker (Future Enhancement)

FROM node:18-alpine
WORKDIR /app
COPY server/package*.json ./
RUN npm ci --only=production
COPY server/ .
EXPOSE 3000
CMD ["npm", "start"]

🧪 Testing

API Test Suite

# Run comprehensive API tests
node test-vector-api.js

The test suite verifies:

Server connectivity and basic endpoints
Vector insertion with proper validation
Similarity search functionality
Health monitoring endpoints
Error handling and validation

Manual Testing

# Health check
curl http://localhost:3000/health

# Server info
curl http://localhost:3000/

# Detailed system metrics
curl http://localhost:3000/health/detailed