DreamRelation: Bridging Customization and Relation Generaion [CVPR 2025]

Qingyu Shi · Lu Qi · Jianzong Wu · Jinbin Bai · Jingbo Wang · Yunhai Tong · Xiangtai Li
PKU, Insta360 Research, NTU, NUS, Shanghai AI

Introduction

In our Relation-Aware Image Customization task, the generated images must accurately preserve the relationships between objects and maintain their identity. We highlight the limitations of previous approaches using three color codes: red indicates failure to capture relationships, blue marks missing objects, and orange represents object confusion. Each image is annotated to reflect its specific issue. Our results, highlighted by green boxes, demonstrate the advantages of our proposed method.

Requirements

Install the packages:

pip install -r requirements.txt

Model Preparation

export HF_ENDPOINT=https://hf-mirror.com
mkdir checkpoints
huggingface-cli download doge1516/MS-Diffusion --local-dir ./checkpoints/MS-Diffusion

Data Preparation

Please prepare datasets and organize them like the following:

├── data
    ├── hug
        ├── videos
            ├── 0
                ├──concept0.png
                ├──concpet1.png
                ├──motion.png
                ├──concept_keyposes_one.pth
                ├──concept_keyposes_two.pth
            ├── 1
            ...
        ├── masks
        ├── prompts.json
    

Training

Please configure your Accelerate settings before running the script. We recommend using two GPUs for training.

accelerate config

Please replace the model and data paths before running the script：

mkdir lora-weights
bash train.sh

Inference

Please replace the model and data paths before running the script, you can also download checkpoints from here：

python inference.py --config config/example_two_objects.yaml

Citation

@inproceedings{DreamRelation,
  title     = {DreamRelation: Bridging Customization and Relation Generation},
  author    = {Shi, Qingyu and Qi, Lu and Wu, Jianzong and Bai, Jinbin and Wang, Jingbo and Tong, Yunhai and Li, Xiangtai},
  booktitle = {CVPR},
  year      = {2025}
}