✨ Highlights

🎞️ A new task: Reselected Key Photo Restoration in Live Photos
Unlike traditional settings, our task addresses a practical scenario where users reselect a new key photo from a Live Photo, which often suffers from degraded quality. We restore a single low-quality frame using the original key photo as a high-quality reference from the same sequence, defining a novel subcategory of reference-based super-resolution (RefSR).

🧩 LiveMoments: A diffusion-based framework for reselected key photo restoration
Our method restores reselected key photos using in-sequence references, with a unified Motion Alignment module to handle spatial misalignment at both latent and image levels. It significantly outperforms existing RefSR and SISR methods in both quantitative metrics and visual quality, even under challenging real-world scenarios.

🔍 Overview

🌟 If LiveMoments is helpful to your projects or Live Photos, please help star this repo. Thanks! :hugs:

⚙️ Dependencies and Installation

# git clone this repository
git clone https://github.com/OpenVeraTeam/LiveMoments.git
cd LiveMoments

# create an environment 
conda create -n livemoments python=3.10
conda activate livemoments
pip install -r requirements.txt

🚀 Quick Start

Step 1: Download the pretrained models

• Download the pretrained SD3 models from HuggingFace.
• Clone the official RAFT repository from GitHub and download the pretrained weights (Sintel).

⚠️ Note: When using RAFT, we recommend commenting out the lines in /path/to/your/RAFT/core/raft.py that normalize the input image tensors to the range [-1, 1].

• Download our pretrained weights from HuggingFace.

You can put the model weights in checkpoint/.

Step 2: Prepare testing data

You can put the testing images in imgs/test. The testing data should consist of three components:

• the reselected key photo
• the original key photo
• the corresponding low-resolution (LR) original key photo, which can be obtained by extracting the corresponding frame from the Live Photo video.

Step 3: Configure paths

Please update the YAML configuration file config/inference_config.yml by specifying the paths to the pretrained model weights, RAFT model, and the testing data.

Ensure that all paths are valid before running inference.

Step 4: Run testing command

python infer/infer_LiveMoments.py \
--output_dir /your/output_dir \
--output_dir_name your_output_dir_name

🖼️ Results

✅ TODO

• Release inference code
• Release pretrained checkpoints
• Release evaluation scripts
• Release test datasets

🎓 Citation

@inproceedings{xue2026livemoments,
  title={LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion},
  author={Clara Xue and Zizheng Yan and Zhenning Shi and Yuhang Yu and Jingyu Zhuang and Qi Zhang and Jinwei Chen and Qingnan Fan},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://openreview.net/forum?id=02mgFnnfqG}
}