EVAL.md
February 13, 2024 ยท View on GitHub
Open-Vocab LVIS evaluation
- for evaluation of our proxydet models, download model first and execute evaluation script referred to below.
| Name | Backbone | Training dataset | mask mAP | mask mAP_novel | Download | Evaluation |
|---|---|---|---|---|---|---|
| BoxSup-R50 | ResNet50 | LVIS | 30.2 | 16.4 | model | - |
| ProxyDet-R50 (wo/ inl) | ResNet50 | LVIS | 30.1 | 19.0 (+2.6) | model | script |
| Detic-R50 | ResNet50 | LVIS + IN-L | 32.4 | 24.9 | model | - |
| ProxyDet-R50 (w/ inl) | ResNet50 | LVIS + IN-L | 32.8 | 26.2 (+1.3) | model | script |
| Detic-SWINB | SWIN-B | LVIS + IN-L | 40.7 | 33.8 | model | - |
| ProxyDet-SWINB (w/ inl) | SWIN-B | LVIS + IN-L | 41.5 | 36.7 (+2.9) | model | script |
- for evaluation on non-pseudo-labeled novel classes, run:
LVIS_INSTASNCE_RESULT_FILE_PATH="YOUR_${LVIS_INSTASNCE_RESULT_FILE_PATH}"
cd tools && python category_wise_ap_lvis.py ${LVIS_INSTASNCE_RESULT_FILE_PATH}
- AP result of
ProxyDet-R50 (w/ inl)on pseudo-labeled novel classes / non-pseudo-labeled novel classes / all novel classes
frequency_group: rare, category_group: in_im
ap: 26.976
frequency_group: rare, category_group: not_in_im
ap: 22.998
frequency_group: rare, category_group: all
ap: 26.216