Metrics Output Files Reference

July 29, 2026 · View on GitHub

This guide describes the output files generated by the metrics system and their formats.

For high-level guidance on checkpoint results and run-level summaries, see:

Checkpoint Results - Complete guide to checkpoint-level metrics and evaluation
Run-Level Results - Complete guide to aggregated run-level metrics

File Locations

Metrics are saved at two levels:

Checkpoint-level (detailed metrics for single checkpoint):

outputs/run_name/problem_name/checkpoint_N/
├── evaluation.json                  # Test results
├── quality_analysis/
│   ├── overall_quality.json         # Snapshot-level aggregates
│   ├── files.jsonl                  # Per-file metrics (one JSON per line)
│   └── symbols.jsonl                # Per-symbol metrics (one JSON per line)
├── diff.json                        # File change statistics
├── rubric.jsonl                     # Rubric grading results
└── ...

Run-level (aggregated across all checkpoints):

outputs/run_name/
├── checkpoint_results.jsonl         # All checkpoint metrics (one per line)
└── result.json                      # Aggregated run summary statistics

overall_quality.json

Aggregated metrics for the entire snapshot. This is the primary file for checkpoint-level analysis.

Structure

{
  "file_count": 12,
  "source_file_count": 8,
  "lines": {
    "total_lines": 1250,
    "loc": 980,
    "comments": 150,
    "multi_comment": 45,
    "single_comment": 105
  },
  "lint": {
    "errors": 5,
    "fixable": 3,
    "counts": {"E501": 2, "F401": 3}
  },
  "symbols": {
    "total": 45,
    "functions": 20,
    "methods": 15,
    "classes": 5,
    "variables": 3,
    "type_aliases": 2,
    "statements": 450,
    "expressions": 890,
    "imports": 35,
    "max_imports": 12,
    "globals": 2
  },
  "functions": {
    "count": 35,
    "cc_sum": 120,
    "cc_max": 15,
    "cc_mean": 3.4,
    "cc_std": 2.1,
    "cc_high_count": 2,
    "cc_extreme_count": 0,
    "depth_max": 4,
    "lines_mean": 12.5,
    "nesting_mean": 1.8,
    "comparisons_mean": 2.3,
    "branches_mean": 1.5,
    "control_mean": 2.1
  },
  "classes": {
    "count": 5,
    "method_counts_mean": 3.0,
    "attribute_counts_mean": 4.2
  },
  "complexity": {
    "cc_ratings": {"A": 28, "B": 5, "C": 2, "D": 0, "E": 0, "F": 0},
    "mi_ratings": {"A": 10, "B": 2, "C": 0},
    "cc_sum": 120,
    "cc_max": 15,
    "num_complex": 2,
    "mi_sum": 245.5,
    "mi_min": 18.2
  },
  "waste": {
    "single_use_functions": 3,
    "trivial_wrappers": 1,
    "single_method_classes": 0
  },
  "graph": {
    "node_count": 8,
    "edge_count": 15,
    "cyclic_dependency_mass": 0.0,
    "propagation_cost": 0.45,
    "dependency_entropy": 0.82
  },
  "source_files": ["main.py", "utils.py", "models.py"]
}

Key Sections

Section	Description
`lines`	Line count aggregates
`lint`	Ruff linting results
`symbols`	Symbol counts by type
`functions`	Function/method statistics
`classes`	Class statistics
`complexity`	CC and MI distributions
`waste`	Abstraction waste counts
`graph`	Dependency graph metrics (optional)

files.jsonl

Per-file metrics in JSON Lines format (one JSON object per line).

Structure (per line)

{
  "file_path": "src/utils.py",
  "symbols": [...],
  "lines": {
    "total_lines": 150,
    "loc": 120,
    "comments": 20,
    "multi_comment": 5,
    "single_comment": 15
  },
  "lint": {
    "errors": 2,
    "fixable": 1,
    "counts": {"E501": 2}
  },
  "mi": 22.5,
  "depth": 1,
  "is_entry_language": true,
  "import_count": 8,
  "global_count": 0,
  "waste": {
    "single_use_functions": [...],
    "trivial_wrappers": [...],
    "single_method_classes": [],
    "single_use_count": 1,
    "trivial_wrapper_count": 0,
    "single_method_class_count": 0
  }
}

Reading files.jsonl

import json

with open("quality_analysis/files.jsonl") as f:
    for line in f:
        file_metrics = json.loads(line)
        print(f"{file_metrics['file_path']}: {file_metrics['lint']['errors']} errors")

symbols.jsonl

Per-symbol (function, method, class) metrics in JSON Lines format.

Structure (per line)

{
  "file_path": "src/utils.py",
  "name": "process_data",
  "type": "function",
  "start": 25,
  "start_col": 0,
  "end": 45,
  "end_col": 0,
  "complexity": 8,
  "branches": 5,
  "statements": 15,
  "expressions_top_level": 10,
  "expressions_total": 25,
  "control_blocks": 3,
  "control_flow": 2,
  "exception_scaffold": 1,
  "comparisons": 4,
  "max_nesting_depth": 3,
  "lines": 20,
  "rating": "B"
}

Symbol Types

Type	Description
`function`	Top-level functions
`method`	Class methods
`class`	Class definitions
`variable`	Module-level variables
`type_alias`	Type alias definitions

diff.json

File change statistics between checkpoints (if available).

Structure

{
  "file_diffs": {
    "src/main.py": {
      "lines_added": 25,
      "lines_removed": 10,
      "status": "modified"
    },
    "src/new_file.py": {
      "lines_added": 50,
      "lines_removed": 0,
      "status": "added"
    }
  },
  "total_added": 75,
  "total_removed": 10
}

rubric.jsonl

LLM-based rubric grading results (if rubric grading was run).

Structure (per line)

{
  "file_path": "src/utils.py",
  "type": "verbosity",
  "criterion": "excessive-comments",
  "line": 45,
  "explanation": "Comment restates obvious code behavior",
  "carried_over": false
}

Or for carried-over grades:

{
  "file_path": "src/utils.py",
  "type": "erosion",
  "criterion": "removed-functionality",
  "line": 30,
  "explanation": "Function no longer handles edge case",
  "carried_over": true,
  "original_checkpoint": 2
}

Using Metrics in the Dashboard

The visualization dashboard (slop_code.visualization.app) reads these files to display:

Checkpoint comparison: Side-by-side metrics across checkpoints
Problem summary: Aggregated metrics per problem
Run overview: Statistics across all problems in a run

Launch the dashboard:

uv run python -m slop_code.visualization.app outputs/

Programmatic Access

Checkpoint-Level Metrics

from slop_code.metrics.checkpoint.driver import get_checkpoint_metrics

# Get all flattened metrics for a checkpoint
metrics = get_checkpoint_metrics(
    checkpoint_dir=Path("outputs/run/problem/checkpoint_1"),
    problem_name="circuit_eval",
    checkpoint_name="checkpoint_1"
)

# Access specific values
print(f"LOC: {metrics['loc']}")
print(f"CC max: {metrics['cc_max']}")
print(f"Verbosity: {metrics['verbosity']}")
print(f"scb-check version: {metrics['scb_check_version']}")
print(f"Pass rate: {metrics['pass_rate']:.1%}")

Quality Metrics for a Directory

from slop_code.metrics.driver import measure_snapshot_quality

# Measure code quality in a directory
snapshot = measure_snapshot_quality(
    dir_path=Path("path/to/code"),
    entry_extensions={".py"}
)

print(f"Files: {snapshot.file_count}")
print(f"Total LOC: {snapshot.lines.loc}")
print(f"CC max: {snapshot.functions.cc_max}")

Run-Level Metrics

import json

# Load run summary
with open("result.json") as f:
    summary = json.load(f)

print(f"Checkpoints solved: {summary['solve_rates']['pct_checkpoints_solved']:.1f}%")
print(f"Total cost: ${summary['costs']['total']:.2f}")

# Stream checkpoint results
with open("checkpoint_results.jsonl") as f:
    for line in f:
        checkpoint = json.loads(line)
        print(f"{checkpoint['problem']} {checkpoint['checkpoint']}: "
              f"{checkpoint['pass_rate']:.0%} pass rate")