CraftGraffiti

Description

Pytorch implementation of the paper CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models. This model is implemented on top of the FLUX framework. The proposed model solves the face consistency issue during style transfer in graffiti art in a two-stage process.

CraftGraffiti transforms a source image into a graffiti-style portrait while preserving the subject’s identity and pose. Graffiti style is injected via a pretrained diffusion fine-tuned with LoRA for the dedicated style. Later on, another diffusion model is equipped with face-consistent self-attention and cross-attention modules to preserve key facial features, and a LoRA module enables pose customization without full model retraining via CLIP-based prompt extension. Finally, multi-scale latent feature processing using a VAE ensures that both global structure and fine details are captured across different resolutions in the latent space, yielding a high-quality graffiti-style image.

Getting Started

Step 1: Clone this repository and change the directory to the repository root

git clone https://github.com/ayanban011/CraftGraffiti.git 
cd CraftGraffiti

Step 2: Setup and activate the conda environment with required dependencies:

conda env create -f craftgraffiti.yml

Step 3: Download the necessary weights:

• IP-Adapter
• Graffiti-LoRA
• Rendered Face Detailer

Step 4: Running

python main1.py
python main.py

Citation

If you find this useful for your research, please cite it as follows:

@misc{banerjee2025craftgraffitiexploringhumanidentity,
      title={CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models}, 
      author={Ayan Banerjee and Fernando Vilariño and Josep Lladós},
      year={2025},
      eprint={2508.20640},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2508.20640}, 
}

Acknowledgement

We have built with the FLUX and IP-Adapter.

Conclusion

Thank you for your interest in our work, and sorry if there are any bugs.