Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation

ICCV, 2023
Xiangtai Li · Haobo Yuan · Wenwei Zhang · Guangliang Cheng
Jiangmiao Pang . Chen Change Loy*

Universal Video Segmentation Model For VSS, VPS, and VIS

News !!

• Update Swin Large Config For Tube-Link VPS on VIP-Seg.

• Code released!!

[Paper] [CODE]

Features

$\color{#2F6EBA}{Universal\ Video\ Segmentation\ Model}$

• A new framework unifies online video segmentation methods and near online video segmentation methods.
• A new unified solution for three video segmentation tasks: VSS, VIS, and VPS.

$\color{#2F6EBA}{Explore\ the\ Cross\-Tube\ Relation}$

• The first video segmentation method that explores the cross-tube relation.
• Proposed Tube-wise matching performs better than Frame-wise matching.

$\color{#2F6EBA}{Strong\ Performance}$

• Achieves strong performance on VIS, VSS, and VPS datasets (five datasets) in one unified architecture.
• Achieves even better performance than those specific architectures.

Dataset

See Dataset.md

Install

See Install.md

Training, Evaluation, and Models

See Train.md

Visualization Results

[VIS] Youtube-VIS 2019

Demo

[VPS] VIP-Seg

Demo

[VSS] VSPW

Demo

[VPS] KITTI-STEP

Demo

Citation

If you think both Tube-Link and its codebase are useful for your research, please consider referring Tube-Link:

@inproceedings{li2023tube,
  title={Tube-link: A flexible cross tube baseline for universal video segmentation},
  author={Li, Xiangtai and Yuan, Haobo and Zhang, Wenwei and Cheng, Guangliang and Pang, Jiangmiao and Loy, Chen Change},
  booktitle={ICCV},
  year={2023}
}

@inproceedings{li2022videoknet,
  title={Video k-net: A simple, strong, and unified baseline for video segmentation},
  author={Li, Xiangtai and Zhang, Wenwei and Pang, Jiangmiao and Chen, Kai and Cheng, Guangliang and Tong, Yunhai and Loy, Chen Change},
  booktitle={CVPR},
  year={2022}
}

License

MIT