BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving

June 10, 2026 · View on GitHub

This repository is the official codebase for BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving.

TLDR: Driving VLAs often generate language reasoning that is useless or even harmful to driving. BLUE addresses this by generating language only when it clearly helps, thereby improving driving performance while reducing inference latency.

BLUE uses a 0.11M-parameter gate to decide at each frame whether to predict driving actions with or without intermediate language generation.

🎉 News

2026-06 - We released the Project Page. It includes some demo videos. 🎉Please check it!

2026-06 - We released the BLUE evaluation code, model checkpoints, and evaluation logs. 🎉Please try it!

⚙️ Environment Setup

Create a Python environment and install the packages listed in requirements.txt.

module load conda
conda create -n blue python=3.8 -y
conda activate blue
python -m pip install -r requirements.txt

Install CARLA 0.9.15 from the official CARLA release page: [CARLA 0.9.15]

After installation, set the CARLA root to the directory that contains:

CarlaUE4.sh
PythonAPI/carla/

📦 Weight Download

The BLUE gate checkpoint is already included in this repository at gate/weights/blue_simlingo_gate.pt.

To use the SimLingo backbone, download the official checkpoint from the official SimLingo repository, then pass the local pytorch_model.pt path through --agent-config when running evaluation.

You can verify bundled assets with:

python scripts/verify_assets.py

Model checkpoints and evaluation logs will also be mirrored on Hugging Face: [Weights] | [Data]

🚀 Quick Start

Static checks

cd blue
module load conda
conda activate blue

bash -n gate/evaluation/eval_blue_full.sh
python scripts/verify_assets.py
python tests/smoke/test_result_summary.py
python tests/smoke/test_gate_checkpoint.py

One-route closed-loop smoke test

cd blue
module load conda
conda activate blue

bash gate/evaluation/eval_blue_full.sh \
  --route-range 0:1 \
  --agent-config /path/to/pytorch_model.pt \
  --carla-root /path/to/carla \
  --out-dir outputs/blue_eval_smoke

📁 Repository Map

blue/
├── data/
│   ├── README.md                         # data release status and layout
│   └── routes/bench2drive_split/         # 220 Bench2Drive route XMLs
├── gate/
│   ├── evaluation/eval_blue_full.sh      # closed-loop evaluation entry point
│   ├── runtime/                          # decision-log utilities
│   └── weights/                          # BLUE gate checkpoint
├── simlingo_training/models/
│   ├── gate.py                           # BLUE gate runtime
│   └── driving_gate.py                   # SimLingo gate integration
├── team_code/agent_simlingo.py           # Bench2Drive agent
├── Bench2Drive/                          # evaluator components
├── evaluation_logs/                      # released evaluation logs
├── configs/                              # asset and evaluation configs
├── docs/                                 # auxiliary notes
├── tests/smoke/                          # smoke tests
└── requirements.txt                      # package snapshot