MODELS.md
July 11, 2023 · View on GitHub
Pre-trained POMP Prompt
| Name (configs) | Model | ImageNet-21K Acc. | Cross-dataset Acc. | Cross-domain Acc. | Epochs | Prompt Length |
|---|---|---|---|---|---|---|
| vit_b16_ep5_randaug2_unc1000_16shots_nctx4_cscFalse_ctpend_seed42.pth.tar | link | 24.9 | 66.7 | 60.4 | 5 | 4 |
| vit_b16_ep20_randaug2_unc1000_16shots_nctx16_cscFalse_ctpend_seed42.pth.tar | link | 25.2 | 65.1 | 60.0 | 20 | 16 |
make a pretrained/ directory under the main directory prompt-pretraining/ and then download the above checkpoints to the pretrained directory. The directory structure should look like:
prompt-pretraining/
|–– pretrained/
| |–– vit_b16_ep5_randaug2_unc1000_16shots_nctx4_cscFalse_ctpend_seed42.pth.tar
| |–– vit_b16_ep20_randaug2_unc1000_16shots_nctx16_cscFalse_ctpend_seed42.pth.tar
Note that the ckpt is only a few KBs since it only contains the pretrained prompt, which is an embedding tensor with size of 4x512 (or 16x512).
POMP for object detection (based on Detic)
(1) Open-vocabulary LVIS
| Name (configs) | Model | mAPr | mAP | Prompt Legth |
|---|---|---|---|---|
| Detic_LbaseI_CLIP_R5021k_640b64_4x_ft4x_max-size_pomp.pth | link | 25.2 | 32.7 | 16 |
(2) Cross-dataset
| Name (configs) | Model | AP50 on LVIS (source) | AP50 on COCO (target) | AP50 on Object365 (target) | Prompt Legth |
|---|---|---|---|---|---|
| Detic_LI_CLIP_R5021k_640b64_4x_ft4x_max-size_pomp.pth | link | 50.7 | 58.0 | 23.4 | 16 |
POMP for semantic segmentation (based on ZSSeg)
(1) Open-vocabulary COCO Stuff
| Name (configs) | Model | hIoU | mIoU-unseen | Prompt Legth |
|---|---|---|---|---|
| coco-stuff-164k-156_zero_shot_proposal_classification_learn_prompt_pomp_bs32_10k.pth | link | 39.1 | 38.2 | 16 |
| coco-stuff-164k-156_zero_shot_maskformer_R101c_pomp_tuned_bs32_60k.pth | link | 39.1 | 38.2 | 16 |
(2) Open-vocabulary Pascal VOC
| Name (configs) | Model | hIoU | mIoU-unseen | Prompt Legth |
|---|---|---|---|---|
| voc-11k-15_zero_shot_proposal_classification_learn_prompt_pomp_bs16_10k.pth | link | 84.4 | 76.8 | 16 |
| voc-11k-15_zero_shot_maskformer_R101c_pomp_tuned_bs16_20k.pth | link | 84.4 | 76.8 | 16 |
(3) Cross-dataset
| Name (configs) | Model | mIoU on COCO Stuff (source) | mIoU on ADE20K (target) | mIoU on PASCAL Context (target) | Prompt Legth |
|---|---|---|---|---|---|
| coco-stuff-164k-171_zero_shot_maskformer_R101c_pomp_tuned_bs32_60k.pth | link | 41.1 | 20.7 | 51.1 | 16 |