EAR: Erasing Concepts from Unified Autoregressive Models

Project Website | ArXiv | Fine-tuned Weights

This repository contains the training, inference, and evaluation code for EAR on unified autoregressive image generation models.

Repository Structure

• train/: EAR, ESD, RACE, and STEREO training scripts.
• infer/: inference scripts for Janus-Pro and Lumina-mGPT.
• eval/: erasure-rate, FID, and CLIP-score evaluation scripts.
• configs/: experiment configs for model and concept pairs.
• data/: train and test prompt files.
• utils/: shared utilities.

Environment

git clone https://github.com/immc-lab/ear.git
cd ear
conda create -n ear python=3.12
conda activate ear
pip install -r requirements.txt

Before running EAR, install the official upstream runtimes in the same environment:

• deepseek-ai/Janus
• Alpha-VLLM/Lumina-mGPT

Training

Janus-Pro:

python -m train.ear_train_janus_pro --config configs/janus_pro_church.yaml
python -m train.ear_train_janus_pro --config configs/janus_pro_nudity.yaml
python -m train.ear_train_janus_pro --config configs/janus_pro_van_gogh.yaml

Lumina-mGPT:

python -m train.ear_train_lumina_mgpt --config configs/lumina_mgpt_church.yaml
python -m train.ear_train_lumina_mgpt --config configs/lumina_mgpt_nudity.yaml
python -m train.ear_train_lumina_mgpt --config configs/lumina_mgpt_van_gogh.yaml

Additional Janus-Pro baselines:

python -m train.esd_train_janus_pro --config configs/janus_pro_church.yaml
python -m train.race_train_janus_pro --config configs/janus_pro_church.yaml
python -m train.stereo_train_janus_pro --config configs/janus_pro_church.yaml

Inference

Janus-Pro:

python -m infer.ear_infer_janus_pro --config configs/janus_pro_church.yaml

Lumina-mGPT:

python -m infer.ear_infer_lumina_mgpt --config configs/lumina_mgpt_church.yaml

Generated results are saved under:

outputs/<model>/<concept>/images/<checkpoint_name>/
  with_erase/
  without_erase/

Evaluation

Church erasure rate:

python -m eval.eval_object --result_dir {result_dir} --output_dir {output_dir} --target_object church

Nudity erasure rate:

python -m eval.eval_nudity --result_dir {result_dir} --output_dir {output_dir}

Van Gogh erasure rate:

python -m eval.eval_style --result_dir {result_dir} --output_dir {output_dir} --classifier_path /path/to/style-classifier --target_style vincent-van-gogh

FID:

python -m eval.eval_fid --help

CLIP score:

python -m eval.eval_clip_score --generated_imgs_dir /path/to/coco30k/generated_imgs/coco30k --csv_path data/coco_30k.csv

Weights

The Finetuned Model Weights are publicly available on Hugging Face.