FlowSim: Optical Flow Similarity Metric for Camera-to-Video Evaluation

FlowSim is a tool for computing optical flow similarity between videos, as introduced in the paper AKiRa: Augmentation Kit on Rays for Optical Video Generation. It provides a robust and scalable metric to evaluate camera motion consistency in generated videos, offering an alternative to traditional pose-based metrics like Absolute Pose Error (APE) and Relative Pose Error (RPE).

---

🧠 Why Optical Flow Similarity?

Traditional metrics such as APE and RPE rely on estimating camera poses, which can be unreliable in scenarios with short baseline (small translation distance) or shaky camera motions and flickering content. Moreover, without knowing the intrinsic parameters, these methods often require scale correction and are slow to compute, hard to parallel, making them less suitable for large-scale evaluations.

FlowSim addresses these limitations by:

• Content Independence: Optical flow focuses on pixel motion, making it less sensitive to scene content variations.
• Robustness to Scale Ambiguities: Unlike pose-based metrics, optical flow does not depend on absolute scale, avoiding issues with scale estimation.
• Computational Efficiency and Scalability: FlowSim leverages GPU-accelerated optical flow estimators like RAFT, significantly reducing computation time compared to Structure-from-Motion (SfM) methods, and easy to parallel.

For a detailed discussion, refer to Section K.2 of the AKiRa supplementary materials.

---

🚀 Getting Started

Usage

python flow_pipeline.py \
  --gen_dir=/path/to/generated/videos \
  --ref_dir=/path/to/reference/videos \
  --num_frames=14 \
  --num_video_to_process=100

Arguments:

• --gen_dir: Directory containing generated video samples to be evaluated.
• --ref_dir: Directory containing reference video clips for optical flow estimation.
• --num_frames: Number of frames to process per video.
• --num_video_to_process: Number of videos to process.

---

📄 Citation

If you use FlowSim or AKiRa in your research, please cite:

@article{wang2024akira,
  title={AKiRa: Augmentation Kit on Rays for optical video generation},
  author={Wang, Xi and Courant, Robin and Christie, Marc and Kalogeiton, Vicky},
  journal={arXiv preprint arXiv:2412.14158},
  year={2024}
}