UCPM: Uncertainty-Guided Cross-Modal Retrieval with Partially Mismatched Pairs

April 16, 2026 · View on GitHub

The official pytorch implementation of UCPM: Uncertainty-Guided Cross-Modal Retrieval with Partially Mismatched Pairs (submitted to IEEE TIP).

Introduction

UCPM framework

Requirements

Python 3.8
PyTorch 1.20.0
numpy
scikit-learn
Punkt Sentence Tokenizer:

import nltk
nltk.download()
> d punkt

(Optional) if the above download failed, you can manually download it from here. The directory structure is:

/home/username/
├── nltk_data
│     ├── tokenizers
│          ├── punkt
│               ├── czech.pickle
│               ├── french.pickle
│               ├── polish.pickle
│               ├── ......

DATASETS

Our directory structure of data.

data
├── f30k_precomp # pre-computed BUTD region features for Flickr30K, provided by SCAN
│     ├── train_ids.txt
│     ├── train_caps.txt
│     ├── ......
│
├── coco_precomp # pre-computed BUTD region features for COCO, provided by SCAN
│     ├── train_ids.txt
│     ├── train_caps.txt
│     ├── ......
│
├── cc152k_precomp # pre-computed BUTD region features for cc152k, provided by NCR
│     ├── train_ids.txt
│     ├── train_caps.tsv
│     ├── ......
│
└── vocab  # vocab files provided by SCAN and NCR
      ├── f30k_precomp_vocab.json
      ├── coco_precomp_vocab.json
      └── cc152k_precomp_vocab.json