Before Start

This document provides a concise workflow to run AuralSAM2 experiments.

⚙️ Prepare environment and data

Please complete all setup steps in installation first.

🚀 Training

Use the unified launcher script:

cd scripts
./run_avs_train.sh <v1s|v1m|v2> [gpus]
./run_ref_train.sh [gpus]

The experiments are implemented by 4 GPUs by default.

🔍 Inference (example)

cd avs.code/v2.code
python inference.py --gpus 1 --batch_size 1 --inference_ckpt /absolute/path/to/checkpoint.pth

📊 Training Logs (Reproducibility)

Some examples of training details, please see this wandb link.

In details, after clicking the run (e.g., v1m-hiera-l), you can checkout:

1. overall information (e.g., command line, hardware information and training time).
2. training curves and validation visualisation.
3. output logs.

💾 Checkpoints

We release both checkpoints and training logs in this Google Drive link.

We also release our checkpoints on Hugging Face 🤗: yyliu01/AuralSAM2. You can download a weight file directly from the repo Files tab, or programmatically with huggingface_hub, for example:

from huggingface_hub import hf_hub_download
ckpt_path = hf_hub_download(
    repo_id="yyliu01/AuralSAM2",
    filename="ckpts/auralsam2_avs_v1m.pth",
)
'''
Available Checkpoints:
- `ckpts/auralsam2_avs_v1m.pth`
- `ckpts/auralsam2_avs_v1s.pth`
- `ckpts/auralsam2_avs_v2.pth`
- `ckpts/auralsam2_refavs_best.pth`
'''