Referral Image Object Segmentation with RefAM

February 2, 2026 · View on GitHub

🔧 Installation

Prerequisites

  • Python 3.10+
  • CUDA 12.1+
  • Conda (recommended)

Setup

conda create -n refam-env python=3.10
conda activate refam-env
pip install -q torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -q diffusers==0.32.2 transformers accelerate einops timm xformers
pip install -q matplotlib pillow numpy opencv-python spacy
pip install -q git+https://github.com/facebookresearch/segment-anything.git
python -m spacy download en_core_web_lg

Download SAM checkpoint (RIOS)

wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth -O checkpoints/sam_vit_h_4b8939.pth

🚀 Quick Start

RIOS Notebook

Notebook

Dataset

  1. HybridGL Data: Follow the setup instructions here.
  2. COCO Captions: Download the annotations here. Then, captions_train2014.json and captions_val2014.json should be placed in the ./refer/data/ directory.

Evaluation on RefCOCO/+/g

$ python eval_concept.py --dataset "refcoco" --split "testA"