LM Vulnerability Judge

October 9, 2025 · View on GitHub

A modular system for analyzing code patches generated by AI agents to detect security vulnerabilities using LLM-as-a-judge.

Overview

This system evaluates code patches from different code agents (like mini-swe-agent, OpenHands) for potential security vulnerabilities including:

  • CWE-538: Insertion of Sensitive Information into Log File
  • CWE-79: Cross-site Scripting (XSS)
  • CWE-89: SQL Injection
  • CWE-94: Code Injection

We collected CWE descriptions from their official website, and prepared them into a CSV file (lm_vulnerability_judge/judges/699.csv).

Features

  • Multi-Judge Support: Support for Qwen 480B and Kimi K2 judges
  • Multiple Agent Support: Supports mini-swe-agent, OpenHands, and SWE-agent output formats
  • LLM-based Assessment: Uses advanced language models for vulnerability detection
  • Multiple Report Formats: JSON and CSV outputs with detailed statistics
  • Configurable: YAML-based configuration for easy customization
  • Extensible: Easy to add new vulnerability types and agents

Basic Setup

  1. Install dependencies:
pip install requests pyyaml openai
  1. Configure your API settings by copying from sample files:
# Copy sample configuration and edit with your API credentials
cp config/qwen-480.yaml.sample config/qwen-480.yaml

Qwen 480B Judge Configuration

Edit config/qwen-480.yaml:

api_base: "https://your-qwen-endpoint/v1"  # Replace with actual endpoint
api_key: "your-qwen-api-key"               # Replace with actual API key
model: "Qwen/Qwen3-Coder-480B-A35B-Instruct"
judge_type: "qwen"
max_tokens: 2048
temperature: 0.1

You also have the choice of using gpt-5-mini. Simply changing model in the config to gpt-5-mini will do.

Basic Usage

# Using Qwen 480B Judge
python run_judge.py config/qwen-480.yaml

# Using Kimi K2 Judge  
python run_judge.py config/kimi-k2.yaml

Test with Limited Patches

python run_judge.py config/kimi-k2.yaml --limit 10 --verbose

Analyze Specific Agent

# Analyze mini-swe-agent patches
python run_judge.py config/qwen-480.yaml --agent mini_swe_agent

# Analyze OpenHands patches  
python run_judge.py config/qwen-480.yaml --agent openhands

# Analyze SWE-agent patches
python run_judge.py config/qwen-480.yaml --agent swe_agent

Override Path (for easier bash script writing)

python run_judge.py config/qwen.yaml --agent ${agent_name//-/_} --preds-path ${preds_path} --reports-path ${reports_path} --cwe-type CWE-${cwe_type} --workers 30

See more details in run_judge.sh.

Project Structure

lm_vulnerability_judge/
├── core/                   # Main system coordination
├── extractors/             # Agent output parsers
│   ├── mini_swe_agent.py  # Mini-SWE-Agent extractor
│   ├── openhands.py       # OpenHands extractor
│   └── swe_agent.py       # SWE-Agent extractor
├── judges/                 # LLM vulnerability assessors
│   ├── qwen_judge.py      # Qwen 480B judge
│   └── kimi_judge.py      # Kimi K2 judge
│   └── gpt5_mini_judge.py # gpt judge
└── aggregators/            # Report generators

config/
├── qwen-480.yaml.sample   # Qwen 480B configuration template
└── kimi-k2.yaml.sample    # Kimi K2 configuration template

vulnerability_reports/      # Generated reports
├── vulnerability_report_*.json
└── vulnerability_summary_*.csv

Configuration

The system uses YAML configuration files. Copy from sample files and edit with your credentials:

cp config/qwen-480.yaml.sample config/qwen-480.yaml
cp config/kimi-k2.yaml.sample config/kimi-k2.yaml

Key configuration settings:

  • api_base: LLM API endpoint
  • api_key: Authentication key
  • model: Model name to use
  • judge_type: Which judge to use ("qwen" or "kimi")
  • max_tokens: Maximum tokens for response
  • temperature: Sampling temperature
  • vulnerabilities: List of CWE types to check
  • agents: Agent-specific configurations
  • output_dir: Where to save reports

Supported Agents

  1. mini-swe-agent: Processes mini-swe-agent output files
  2. openhands: Processes OpenHands experiment results
  3. swe-agent: Processes SWE-agent predictions and reports

Output

The system generates:

  1. JSON Report: Detailed vulnerability assessments for each patch
  2. CSV Summary: High-level statistics and findings
  3. Console Output: Real-time progress and summary statistics

Adding New Agents

  1. Create a new extractor in extractors/
  2. Inherit from BaseExtractor
  3. Implement extract_patches() and extract_functional_tests()