AST Copilot Helper

October 10, 2025 ยท View on GitHub

A comprehensive toolkit for Abstract Syntax Tree analysis and Model Context Protocol (MCP) server implementation, enabling AI agents to understand and query codebases with semantic intelligence across 15 programming languages.

๐Ÿš€ AI-Powered Code Understanding

Transform your codebase into an AI-accessible knowledge base:

  • ๐Ÿ” Semantic Search: Query code using natural language
  • ๐Ÿค– MCP Integration: Enable AI agents to understand your code structure
  • โšก Performance: Fast parsing with intelligent caching and incremental updates
  • ๐ŸŒ Multi-Language:Extract semantic information from your codebase:
# Initialize configuration and database
# Automatically creates .gitignore to exclude .astdb/ directory
yarn ast-copilot-helper init

# Skip .gitignore creation if you have custom version control setup
yarn ast-copilot-helper init --no-gitignore

# Parse a directory
yarn ast-copilot-helper parse src/

# Parse with natural language query
yarn ast-copilot-helper query "functions that handle authentication"

Note: The init command automatically creates or updates .gitignore to exclude the .astdb/ directory (database files, vector indexes, models, cache) from version control. This prevents accidentally committing large generated files. Use --no-gitignore if you prefer custom gitignore handling. 15 programming languages across 3 tiers

  • ๐Ÿ“ฆ Snapshot Distribution: Share pre-built databases for instant team onboarding

๐Ÿข Tier 1: Core Languages (4 languages)

JavaScript โ€ข TypeScript โ€ข Python โ€ข Rust

Java โ€ข C++ โ€ข C โ€ข C# โ€ข Go โ€ข Ruby โ€ข PHP

๐ŸŽฏ Tier 3: Specialized Languages (4 languages)

Kotlin โ€ข Swift โ€ข Scala โ€ข Bash

Powered by tree-sitter 0.25.x ecosystem

๐Ÿ“š Complete Language Guide โ†’

Prerequisites

  • Node.js: 20.0.0 or higher
  • Yarn: 4.x or higher (Berry - modern Yarn)
  • Operating Systems: Windows, macOS, Linux (x64, arm64)

Quick Installation

Clone and set up the project:

git clone https://github.com/EvanDodds/ast-copilot-helper.git
cd ast-copilot-helper
yarn install

This automatically:

  • Installs all dependencies using Yarn workspaces
  • Links packages together
  • Builds all components

Overview

AST Copilot Helper bridges the gap between your codebase and AI agents by providing semantic understanding through Abstract Syntax Tree analysis. The toolkit consists of three integrated components:

  • @ast-copilot-helper/ast-helper - Core CLI tool that parses source code and builds semantic databases
  • @ast-helper/core-engine - High-performance Rust engine (WASM for vector ops, native for AST processing)
  • @ast-copilot-helper/ast-mcp-server - Model Context Protocol server enabling AI agents to query code semantically
  • @ast-copilot-helper/vscode-extension - VS Code extension for seamless integration (optional)

Architecture

ast-copilot-helper/                 # Monorepo root
โ”œโ”€ packages/
โ”‚  โ”œโ”€ ast-helper/                   # ๐Ÿ”ง Core TypeScript library
โ”‚  โ”‚  โ”œโ”€ src/                       # TypeScript parsing & analysis
โ”‚  โ”‚  โ””โ”€ dist/                      # Compiled output
โ”‚  โ”œโ”€ ast-core-engine/              # โšก High-performance Rust engine
โ”‚  โ”‚  โ”œโ”€ src/                       # Rust source (WASM + native bindings)
โ”‚  โ”‚  โ”œโ”€ pkg/                       # Build output (WASM modules)
โ”‚  โ”‚  โ”œโ”€ target/                    # Rust build artifacts
โ”‚  โ”‚  โ””โ”€ Cargo.toml                 # Rust configuration
โ”‚  โ”œโ”€ ast-mcp-server/               # ๐Ÿค– MCP protocol server
โ”‚  โ”‚  โ”œโ”€ src/                       # TypeScript source
โ”‚  โ”‚  โ”œโ”€ bin/ast-mcp-server         # Server executable
โ”‚  โ”‚  โ””โ”€ dist/                      # Compiled output
โ”‚  โ””โ”€ vscode-extension/             # ๐ŸŽจ VS Code extension
โ”‚     โ”œโ”€ src/                       # TypeScript source
โ”‚     โ””โ”€ dist/                      # Compiled output
โ”œโ”€ docs/                            # ๐Ÿ“š Documentation
โ”‚  โ”œโ”€ guide/                        # User guides
โ”‚  โ”œโ”€ api/                          # API references
โ”‚  โ””โ”€ examples/                     # Usage examples
โ”œโ”€ tests/                           # ๐Ÿงช Test suites
โ”‚  โ”œโ”€ integration/                  # Integration tests
โ”‚  โ”œโ”€ fixtures/                     # Test repositories
โ”‚  โ””โ”€ benchmarks/                   # Performance tests
โ””โ”€ scripts/                         # ๐Ÿ”ง Build & maintenance

๐ŸŒ Language Support

AST Copilot Helper supports 15 programming languages organized into 3 tiers:

TierLanguagesUse Cases
๐Ÿข CoreJavaScript, TypeScript, Python, RustFoundation languages, highest optimization
๐Ÿ‘ฉโ€๐Ÿ’ป PopularJava, C++, C, C#, Go, Ruby, PHPWidely-used production languages
๐ŸŽฏ SpecializedKotlin, Swift, Scala, BashDomain-specific and emerging technologies

Quick Example

import { ParserFactory } from "@ast-copilot-helper/ast-helper";

// Create parser for any supported language
const parser = await ParserFactory.createParser();

// Parse TypeScript
const tsResult = await parser.parseCode(
  'function hello(): string { return "world"; }',
  "typescript",
);

// Parse Python
const pyResult = await parser.parseCode(
  'def hello() -> str:\n    return "world"',
  "python",
);

// Parse Go
const goResult = await parser.parseCode(
  'func hello() string {\n    return "world"\n}',
  "go",
);

๐Ÿ“– Complete Language Guide โ€ข โšก Performance Benchmarks โ€ข ๐Ÿ”ง API Reference

๐Ÿ“ฆ Snapshot Distribution System

Share pre-built .astdb databases to reduce onboarding time from hours to minutes:

# Create and publish a snapshot
yarn ast-helper snapshot create --snapshot-version 1.0.0 --description "Production snapshot"
yarn ast-helper snapshot publish snapshot-1.0.0.tar.gz

# On another machine, download and restore
yarn ast-helper snapshot download 1.0.0
yarn ast-helper snapshot restore snapshot-1.0.0.tar.gz
# Ready to work instantly!

Key Features

  • โšก Instant Setup: Skip parsing, restore pre-built databases in seconds
  • ๐ŸŒ Team Collaboration: Share snapshots via GitHub Releases or custom backends
  • ๐Ÿ”„ CI/CD Integration: Automated snapshot creation and publishing
  • ๐Ÿ“ Versioning: Semantic versioning with tags and metadata
  • ๐Ÿ” Validation: Checksum verification and backup creation
  • ๐Ÿ“Š Compression: Configurable compression (0-9) for optimal size/speed

Use Cases

  • New Developer Onboarding: Instant access to parsed codebase
  • CI/CD Pipelines: Skip parsing in every pipeline run
  • Team Synchronization: Ensure everyone has identical database states
  • Release Snapshots: Reproducible database states for each version

Automated Snapshots

The project includes GitHub Actions workflow for automated snapshot creation:

  • Push to Main: Creates latest snapshot after code changes
  • Nightly Schedule: Daily snapshots at 2 AM UTC
  • Release Events: Tagged snapshots for each release
  • Manual Trigger: On-demand snapshot creation

๐Ÿ“š Complete Snapshot Guide โ†’

๐Ÿค– MCP Server Integration

The AST Copilot Helper includes a full-featured Model Context Protocol (MCP) server, enabling AI agents like Claude Desktop, Cline, and other MCP-compatible tools to query your codebase semantically.

Quick Start with MCP

# Start the MCP server
yarn ast-mcp-server start

# Or use with Claude Desktop
# Add to Claude Desktop config (~/.config/Claude/config.json):
{
  "mcpServers": {
    "ast-helper": {
      "command": "node",
      "args": ["/path/to/ast-copilot-helper/packages/ast-mcp-server/dist/index.js"],
      "cwd": "/path/to/your/project"
    }
  }
}

MCP Server Configuration

Configure the MCP server port and auto-start behavior in your config file:

{
  "mcp": {
    "port": 3000,
    "autoStart": false
  }
}

Configuration Options:

  • port (number, default: 3000): Server port number
    • Valid range: 1024-65535 (unprivileged ports)
    • Automatically validated during configuration loading
  • autoStart (boolean, default: false): Auto-start server on initialization
    • When true, the MCP server starts automatically after initialization
    • When false, the server must be started manually with yarn ast-mcp-server start

Example Configurations:

// Development: Manual start, custom port
{
  "mcp": {
    "port": 8080,
    "autoStart": false
  }
}

// Production: Auto-start on default port
{
  "mcp": {
    "port": 3000,
    "autoStart": true
  }
}

See examples/config.json for a complete configuration example.

User Configuration

AST Copilot Helper supports user-level configuration following the XDG Base Directory Specification, allowing you to set default preferences that apply across all projects.

Default User Config Path:

  • $XDG_CONFIG_HOME/ast-copilot-helper/config.json (if XDG_CONFIG_HOME is set)
  • ~/.config/ast-copilot-helper/config.json (fallback)

Custom User Config Path:

ast-helper --user-config=/path/to/config.json <command>

Configuration Priority (highest to lowest):

  1. CLI arguments (--top-k, --batch-size, etc.)
  2. Environment variables (AST_COPILOT_*)
  3. Project config (.astdb/config.json in workspace)
  4. User config (XDG or custom path)
  5. Built-in defaults

Example User Config (~/.config/ast-copilot-helper/config.json):

{
  "topK": 15,
  "snippetLines": 7,
  "enableTelemetry": false,
  "model": {
    "showProgress": true
  },
  "mcp": {
    "port": 3000,
    "autoStart": false
  }
}

Use Cases:

  • Personal Preferences: Set your preferred topK and snippetLines values
  • Privacy: Disable telemetry by default in user config
  • Development Setup: Configure MCP server port for your local environment
  • Team Collaboration: Project config (.astdb/config.json) overrides user config for team standards

Creating User Config:

# Create config directory
mkdir -p ~/.config/ast-copilot-helper

# Create config file
cat > ~/.config/ast-copilot-helper/config.json << 'EOF'
{
  "topK": 20,
  "enableTelemetry": false
}
EOF

Cross-Platform Support:

  • Linux/macOS: Uses XDG Base Directory Specification
  • Windows: Falls back to ~/.config (works with WSL and native Windows)
  • Environment Override: Set XDG_CONFIG_HOME to customize location

Available MCP Tools

The server exposes these tools for AI agents:

ToolDescription
query_ast_contextSemantic search across your codebase
ast_file_queryAnalyze specific files with AST parsing
ast_index_statusGet indexing status and statistics
ast_find_referencesFind all references to a symbol
ast_find_definitionsLocate symbol definitions
ast_symbol_searchSearch for symbols by name or pattern

VS Code Extension Integration

The VS Code extension provides seamless MCP client integration:

  • Automatic Server Management: Extension spawns and manages the MCP server process
  • Custom Transport: Uses managed process with sophisticated lifecycle handling
  • Connection Monitoring: Auto-reconnect, heartbeat, health checks
  • Status Bar Integration: Real-time connection status indicator
  • Output Channel: Detailed logs for debugging

Extension Features

// The extension uses real MCP SDK with custom Transport
import { Client } from "@modelcontextprotocol/sdk/client/index.js";

// Custom Transport wraps managed process
class ManagedProcessTransport implements Transport {
  // Handles JSON-RPC over stdin/stdout
  // Preserves VS Code extension's process management
}

// Client connects to server
await client.connect(transport);
const result = await client.callTool({
  name: "query_ast_context",
  arguments: { query: "find all functions" },
});

Architecture Benefits

Why Custom Transport?

  • Preserves VS Code extension patterns (extensions manage their servers)
  • Enables custom monitoring, logging, and UI integration
  • Allows sophisticated restart logic and health checks
  • MCP server remains usable by other tools (Claude Desktop, Cline)

Process Lifecycle

ServerProcessManager โ†’ Spawns & monitors MCP server
     โ†“
ManagedProcessTransport โ†’ Wraps process stdio for JSON-RPC
     โ†“
MCP SDK Client โ†’ Standard protocol communication
     โ†“
AI Agent โ†’ Queries codebase semantically

Use Cases

  • Claude Desktop: Chat with your codebase using Claude
  • VS Code Extension: Integrated code understanding in your editor
  • Cline: AI-powered code generation with context
  • Custom Integrations: Build your own MCP-compatible tools

๐Ÿ“– MCP Testing Guide โ†’ โ€ข ๐Ÿ”ง Integration Status โ†’

๐Ÿ”’ Security Features

AST Copilot Helper includes comprehensive security features for model verification and secure downloads:

Model Verification & Security

  • โœ… SHA256 Checksum Verification: Automatic integrity validation for all downloaded models
  • ๐Ÿ” Digital Signature Verification: RSA/ECDSA signature validation for model authenticity
  • ๐Ÿ›ก๏ธ Security Hooks System: Extensible pre/post download verification framework
  • ๐Ÿ“‹ Comprehensive Audit Logging: Security event tracking with JSONL-based audit trails
  • โš ๏ธ Automatic Quarantine: Suspect files automatically quarantined for investigation

Security Components

import {
  SignatureVerifier,
  SecurityHooksManager,
  securityLogger,
  SecurityEventType,
} from "@ast-copilot-helper/ast-helper";

// Initialize signature verification
const verifier = new SignatureVerifier();
await verifier.initialize();

// Add trusted public key
await verifier.addPublicKey("model-provider-key", publicKeyPem, {
  keyId: "model-provider-key",
  algorithm: "RSA",
  issuedBy: "Model Provider Inc",
  validUntil: new Date("2026-12-31"),
});

// Security hooks automatically validate downloads
const hooksManager = new SecurityHooksManager();

// Register custom security hook
hooksManager.registerHook(HookType.PRE_DOWNLOAD, async (context) => {
  // Custom validation logic
  return {
    allowed: true,
    errors: [],
    warnings: [],
  };
});

// Query security audit log
const recentEvents = await securityLogger.getRecentEvents(100, {
  type: SecurityEventType.VERIFICATION_FAILED,
  severity: SecuritySeverity.ERROR,
});

Default Security Checks:

  • HTTPS URL validation (prevents insecure downloads)
  • Metadata validation (ensures required fields)
  • File integrity verification (SHA256 checksums)
  • Digital signature verification (when signatures provided)

๐Ÿ“š Security Guide โ€ข ๐Ÿ” API Reference

โšก Performance & Architecture

AST Copilot Helper uses a native-first architecture combining TypeScript flexibility with Rust performance:

Hybrid Engine Architecture

AST Copilot Helper uses a hybrid Rust architecture optimized for both performance and ease of distribution:

ComponentTechnologyDistributionRationale
Vector DatabaseRustWASM (universal)One binary for all platforms
AST ProcessingRustNative (if needed)Maximum performance for parsing
TypeScript CoreTypeScriptUniversalCross-platform business logic

Key Performance Features

  • ๐Ÿš€ Rust-Powered Operations: High-performance vector similarity search and AST processing
  • ๐Ÿ“ฆ Smart Distribution: WASM for universal compatibility, native where performance critical
  • ๐ŸŒ Zero Install Friction: WASM eliminates platform-specific binary issues
  • ๐Ÿ”„ Smart Language Detection: Intelligent grammar loading and caching
  • ๐Ÿ’พ Incremental Processing: Smart cache invalidation and differential updates

Current Performance Characteristics:

  • AST parsing: 1-50ms depending on language tier and file size
  • Vector search: High-performance Rust operations (compiled to WASM for universal deployment)
  • Batch processing: 5000+ files with intelligent memory management
  • Language support: 15 languages across 3 performance tiers

Architecture Note: Vector database uses Rust compiled to WebAssembly for universal compatibility. While WASM adds ~10-30% overhead vs native, it eliminates platform-specific build complexity and ensures "npm install just works" everywhere.

CI/CD Pipeline

This project implements a comprehensive CI/CD pipeline with 36 acceptance criteria across 6 categories:

๐Ÿš€ Pipeline Features

  • Multi-platform builds: Windows, macOS, Linux with Node.js 20, 22, 24
  • Comprehensive testing: Unit, integration, and performance tests
  • Quality gates: 90%+ coverage, security scanning, performance validation
  • Blue-Green deployment: Zero-downtime staging and production deployments
  • Real-time monitoring: Performance tracking, alerting, and interactive dashboards
  • Multi-channel notifications: Slack, email, and GitHub integration

๐Ÿ“Š Monitoring & Alerts

  • Performance monitoring with trend analysis and A-F grading
  • Real-time dashboards with Chart.js visualization and auto-refresh
  • Intelligent alerting with escalation rules and cooldown management
  • Build failure notifications with multi-channel delivery

๐Ÿ”ง CI/CD Commands

# Quality gates
yarn run ci:quality-gate
yarn run ci:security-scan
yarn run ci:performance-score

# Deployment
yarn run ci:deploy-staging
yarn run ci:deploy-production
yarn run ci:health-check
yarn run ci:rollback

# Monitoring
yarn run ci:performance-monitor
yarn run ci:monitoring-dashboard
yarn run ci:alerting-system

For complete CI/CD documentation, see docs/CI-CD-PIPELINE.md.

Development

Building

Build all packages:

yarn run build

Build and watch for changes:

yarn run build:watch

Testing

We have a multi-tiered testing strategy optimized for both development speed and comprehensive validation:

Testing Strategy

We use a multi-tiered testing approach optimized for different development phases:

Git Hook Testing (Automated)

  • Pre-commit (~30-45 seconds): Essential unit tests + type checking + linting
    • Runs fastest subset to catch basic issues before commit
    • Prevents broken code from entering the repository
  • Pre-push (~1-2 minutes): Comprehensive fast tests + build verification
    • All tests except performance benchmarks
    • Ensures code is ready to be shared/reviewed

Manual Testing Commands

# Essential unit tests only (fastest, used by pre-commit)
yarn run test:unit

# Fast comprehensive tests (used by pre-push, excludes benchmarks)
yarn run test:fast
yarn run test:precommit  # Alias for test:fast

# Integration and end-to-end tests
yarn run test:integration

# Complete test suite including performance benchmarks
yarn run test:all  # Takes 5-7 minutes, use sparingly

Performance & Benchmarks

# Performance benchmarks (slowest tests)
yarn run test:benchmarks

Comprehensive Testing

# Run ALL tests (unit + integration + benchmarks) - use for thorough validation
yarn run test:all
yarn run test:dev        # Same as above
yarn run test:comprehensive  # Same as above

With Coverage

yarn run test:coverage

Git Hook Strategy:

  • Pre-commit: Fast tests only (type-check, lint, fast unit tests, build)
  • Pre-push: Comprehensive tests (unit + integration, skip benchmarks)
  • Manual/CI: All tests including performance benchmarks

Type Checking

Check TypeScript types across all packages:

yarn run typecheck

Package-Specific Commands

You can run commands in specific packages:

# Run commands in ast-copilot-helper package
cd packages/ast-copilot-helper
yarn run build
yarn run dev
yarn test

# Run commands in ast-mcp-server package
cd packages/ast-mcp-server
yarn run build
yarn run dev
yarn test

# Run commands in vscode-extension package
cd packages/vscode-extension
yarn run build
yarn run dev
yarn test

Security Framework

This project includes a comprehensive security framework designed to protect against common vulnerabilities and ensure secure AST processing workflows.

Security Features

  • ๐Ÿ”’ Comprehensive Security Auditing: Multi-layer security analysis with OWASP, CWE, and NIST compliance
  • ๐Ÿ›ก๏ธ Input Validation System: Advanced input sanitization and validation with XSS, SQL injection, and path traversal protection
  • ๐Ÿ” Vulnerability Scanning: Automated detection of security patterns, hardcoded credentials, and insecure cryptographic practices
  • ๐Ÿ”ง Security Hardening Framework: Policy enforcement, access control, and security configuration management
  • ๐Ÿ“Š Security Integration Testing: End-to-end security workflow validation with performance monitoring

Security Components

The security framework consists of four core modules:

  1. ComprehensiveSecurityAuditor (packages/ast-copilot-helper/src/security/auditor.ts)

    • Performs comprehensive security audits with OWASP/CWE/NIST compliance
    • Generates detailed security reports with risk scoring and remediation guidance
    • Supports dependency vulnerability scanning and policy enforcement
  2. ComprehensiveInputValidator (packages/ast-copilot-helper/src/security/input-validator.ts)

    • Advanced input validation and sanitization engine
    • Protection against XSS, SQL injection, command injection, and path traversal
    • Context-aware validation with custom rule support
  3. VulnerabilityScanner (packages/ast-copilot-helper/src/security/vulnerability-scanner.ts)

    • Pattern-based vulnerability detection for common security issues
    • Supports hardcoded credential detection, insecure crypto practices, and injection vulnerabilities
    • Real-time risk scoring and finding categorization
  4. SecurityHardeningFramework (packages/ast-copilot-helper/src/security/security-hardening-framework.ts)

    • Security policy enforcement and configuration management
    • Access control validation and permission management
    • Security baseline compliance checking

Security Testing

Comprehensive test coverage ensures security framework reliability:

  • 139+ Unit Tests: Core security functionality validation (100% success rate)
  • 14 Integration Tests: End-to-end security workflow testing
  • Real-world Scenarios: SQL injection, XSS, cryptographic vulnerability testing
  • Performance Validation: Security operations under load testing

Run security tests:

# All security unit tests
yarn test packages/ast-copilot-helper/src/security/

# Integration security tests
yarn test tests/integration/security-integration.test.ts

# Complete security test suite
yarn run test:security

Security Configuration

Security settings are configured via the security config system:

import { DEFAULT_SECURITY_CONFIG } from "packages/ast-copilot-helper/src/security/config";

// Default security configuration includes:
// - OWASP/CWE/NIST compliance frameworks
// - Input validation rules and sanitization
// - Vulnerability detection patterns
// - Security hardening policies

For detailed security information, see SECURITY.md.

Usage

Quick Start

1. Parse Your Code

Extract semantic information from your codebase:

# Initialize configuration and database
# Automatically creates .gitignore to exclude .astdb/ directory
yarn ast-copilot-helper init

# Skip .gitignore creation if you have custom version control setup
yarn ast-copilot-helper init --no-gitignore

# Parse a directory
yarn ast-copilot-helper parse src/

# Parse with natural language query
yarn ast-copilot-helper query "functions that handle authentication"

Note: The init command automatically creates or updates .gitignore to exclude the .astdb/ directory (database files, vector indexes, models, cache) from version control. This prevents accidentally committing large generated files. Use --no-gitignore if you prefer custom gitignore handling.

Git Integration for Smart Parsing

Process only the files you care about using git-aware modes:

# Pre-commit validation - parse only staged files
yarn ast-copilot-helper parse --staged

# PR validation - parse files changed since main branch
yarn ast-copilot-helper parse --base main

# Development workflow - parse all working directory changes
yarn ast-copilot-helper parse --changed

# Feature branch comparison - parse changes since develop
yarn ast-copilot-helper parse --base origin/develop

Use Cases:

  • Pre-commit hooks: --staged ensures only staged files are validated before commit
  • CI/CD pipelines: --base main validates only files changed in PR
  • Local development: --changed provides quick feedback on modified code
  • Branch reviews: --base <branch> compares against any git reference

2. Watch for Changes

Monitor your codebase for changes with intelligent incremental updates:

# Basic watch mode (parse only)
yarn ast-copilot-helper watch

# Full pipeline: parse โ†’ annotate โ†’ embed
yarn ast-copilot-helper watch --full-pipeline

# Parse and annotate only (no embedding)
yarn ast-copilot-helper watch --full-pipeline --no-embed

# Watch with custom glob pattern
yarn ast-copilot-helper watch --glob "src/**/*.ts"

# Optimize for large codebases
yarn ast-copilot-helper watch --batch-size 100 --debounce 500

Key Features:

  • Intelligent Skip Detection: Automatically skips files with unchanged content using SHA256 hashing
  • Content-Based Rename Detection: Detects renamed/moved files using content hashes without re-parsing
  • Cross-Directory Moves: Tracks file moves across directories (e.g., src/old.ts โ†’ lib/new.ts)
  • Two-Pass Analysis: First detects deletions, then matches additions to prevent false positives
  • Persistent State: Resumes from crash with .astdb/watch-state.json
  • Pipeline Integration: Optional automatic annotation via Rust CLI and embedding
  • Performance Statistics: Tracks files processed, skipped, errors, and timing

Rename Detection Workflow:

  1. Pass 1 - Deletions: Identify deleted files and store their content hashes
  2. Pass 2 - Additions: Match new files against deleted file hashes
  3. Rename Window: 5-second window to match renames (configurable)
  4. Database Update: Update paths in database instead of re-parsing

Performance Impact:

  • Without Rename Detection: ~500ms per file (full re-parse)
  • With Rename Detection: ~50ms per file (database path update)
  • Savings: ~90% faster for renamed/moved files

Use Cases:

  • Development Workflow: Automatic parsing as you code with instant feedback
  • Long-Running Sessions: Stable memory usage with auto-save and crash recovery
  • CI/CD Integration: Watch mode for continuous validation during development
  • Large Codebases: Smart batching and skip detection for optimal performance

3. Model Verification & Registry

Manage and verify ONNX embedding models with built-in security:

# Register a new model in the registry
yarn ast-helper model register my-model \\
  --version 1.0.0 \\
  --url https://example.com/model.onnx \\
  --checksum sha256:abc123... \\
  --format onnx

# Verify model integrity (checksum, format, optional signature)
yarn ast-helper model verify my-model

# List all registered models
yarn ast-helper model list

# View model details and verification history
yarn ast-helper model info my-model

# Delete a model from registry
yarn ast-helper model delete my-model

Key Features:

  • ๐Ÿ” SHA256 Checksum Verification: Automatic integrity validation on download and use
  • โœ๏ธ Digital Signature Verification: Optional RSA/ECDSA signature validation for authenticity
  • ๐Ÿ“ Verification History: Complete audit trail of all verification attempts
  • ๐Ÿ“Š Statistics Dashboard: Track verification success rates and model health
  • ๐Ÿ—„๏ธ SQLite Registry: Persistent model metadata storage

Verification Workflow:

  1. Model is registered with URL, checksum, and optional signature
  2. On first use, model is downloaded and verified (checksum + signature)
  3. Verification status is recorded in registry with timestamp
  4. Subsequent uses check if re-verification is needed (configurable interval)
  5. Failed verifications are logged with detailed error messages

4. Query Cache Management

Optimize query performance with multi-level caching:

# Warm the cache with frequently used queries
yarn ast-helper cache warm --top 50

# Analyze cache performance and hit rates
yarn ast-helper cache analyze

# Prune old or least-used cache entries
yarn ast-helper cache prune --max-age 30 --max-entries 1000

# Clear all caches (L1 memory, L2 disk, L3 database)
yarn ast-helper cache clear

# View cache statistics
yarn ast-helper cache stats

Cache Architecture:

  • L1 (Memory): Hot queries, ~100ms access time, LRU eviction
  • L2 (Disk): Recent queries, ~500ms access time, size-based eviction
  • L3 (Database): Historical queries, ~2s access time, TTL-based cleanup

Key Features:

  • ๐Ÿ“ˆ Automatic Promotion: L3 hits promoted to L2, L2 hits promoted to L1
  • ๐Ÿ”„ Smart Invalidation: Automatic cache invalidation on file/index changes
  • ๐Ÿ“Š Query Analytics: Track query frequency, execution time, and cache hit rates
  • ๐ŸŽฏ Cache Warming: Pre-populate cache with frequently used queries
  • โšก Performance Gains: Up to 10x faster on cached queries

Use Cases:

  • Production Optimization: Pre-warm cache with common queries before deployment
  • Development Speed: Cache warm during dev for instant query responses
  • Resource Management: Prune old cache entries to manage disk space
  • Performance Monitoring: Analyze cache hit rates and query patterns

5. Start MCP Server

Enable AI agent integration:

# Start the MCP server
yarn ast-mcp-server --port 3000

# Or use VS Code extension for integrated experience
code --install-extension ast-copilot-helper

3. Explore Documentation

Technical Foundation

  • ๐Ÿ—๏ธ Monorepo: Yarn v4 workspaces with TypeScript project references
  • โšก TypeScript: Strict configuration targeting ES2022 with full type safety
  • ๏ฟฝ Rust Core: High-performance native engine via NAPI bindings
  • ๏ฟฝ๐Ÿงช Testing: Comprehensive test suite with Vitest (unit, integration, benchmarks)
  • ๐Ÿ”„ CI/CD: Automated testing, quality gates, and deployment pipeline with binary releases
  • ๐ŸŒ Cross-Platform: Full Windows, macOS, and Linux support (x64, arm64)

Current Status

AST Copilot Helper is production-ready with:

  • โœ… Complete AST processing for 15 programming languages
  • โœ… Functional MCP server with semantic query capabilities
  • โœ… VS Code extension with integrated workflow
  • โœ… Comprehensive security framework with vulnerability scanning
  • โœ… Advanced CI/CD pipeline with monitoring and automated deployments
  • โœ… 188 unit tests, 187 integration tests, 207 benchmark tests (all passing)

Community & Support

๐Ÿ“‹ Getting Help

๐Ÿ“– Community Resources

Contributing

We welcome contributions! See our Contributing Guide for comprehensive details.

Quick Contributor Setup

# 1. Fork and clone
git clone https://github.com/YOUR_USERNAME/ast-copilot-helper.git
cd ast-copilot-helper

# 2. Install and build
yarn install
yarn build

# 3. Run tests
yarn test:fast        # Quick validation
yarn test:all         # Full test suite

# 4. Create feature branch and submit PR
git checkout -b feature/your-feature
# ... make changes ...
git commit -m "feat: your feature"
git push origin feature/your-feature

๐Ÿ“‹ Use our templates: Issues โ€ข Pull Requests
๐Ÿ’ฌ Join discussions: GitHub Discussions
โ“ Get help: Support Guide
๐Ÿš€ For maintainers: Release Process Guide

License & Security


๐Ÿ“š Full Documentation โ€ข ๐Ÿš€ Getting Started โ€ข ๐Ÿค Contributing