Peformance Comparison of Cross-modal Retrieval

Catalogue

• Commonly-used datasets
  • Flickr8K
  • Flickr30K
  • MSCOCO1K
  • MSCOCO5K
• Identity-aware datasets
  • RSTPReid
  • CUHK-PEDES
  • ICFG-PEDES
  • CUB-Flowers

Peformance of Commonly-used Datasets

Performance of Flickr8K

(* indicates Ensemble models, ^ indicates questionable authen)

Method_name	Concise_note	Sentence retrieval			Image retrieval
		R@1	R@5	R@10	R@1	R@5	R@10
DeViSE	RCNN	4.8	16.5	27.3	5.9	20.1	29.6
SDT-RNN	AlexNet	4.5	18.0	28.6	6.1	18.5	29.0
SDT-RNN	RCNN	6.0	22.7	34.0	6.6	21.6	31.7
DeFrag	AlexNet	5.9	19.2	27.3	5.2	17.6	26.5
DeFrag	RCNN	12.6	32.9	44.0	9.7	29.6	42.5
m-RNN	AlexNet	14.5	37.2	48.5	11.5	31.0	42.4
DVSA	DepTree	14.8	37.9	50.0	11.6	31.4	43.8
DVSA	RCNN	16.5	40.6	54.2	11.8	32.1	44.7
UVSE	AlexNet	13.5	36.2	45.7	10.4	31.0	43.7
UVSE	VggNet	18.0	40.9	55.0	12.5	37.0	51.5
NIC	GoogleNet	20	--	61	19	--	64
m-CNN*	OverFeat	14.9	35.9	49.0	11.8	34.5	48.0
m-CNN*	VggNet	24.8	53.7	67.1	20.3	47.6	61.7
HM-LSTM	RCNN	27.7	--	68.6	24.4	--	68.1
SPE	VggNet	30.1	60.4	73.7	23.0	51.3	64.8
FV	GMM+HGLMM	31.0	59.3	73.7	21.2	50.0	64.8
MFM	VggNet	35.6	67.0	78.6	28.4	58.5	72.3
NAA	ResNet	37.2	68.1	79.1	27.7	59.6	71.8
ITMeetsAL	MobileNet	30.9	58.6	70.8	--	--	--
ITMeetsAL	ResNet	40.1	67.8	79.2	--	--	--
2WayNet	VggNet	43.4	63.2	--	29.3	49.7	--
SCAN*	BUTD	52.2	81.0	89.2	38.3	67.8	78.9
IMRAM	BUTD, Image	48.5	78.1	85.3	32.0	61.4	73.9
IMRAM	BUTD, Text	52.1	81.5	90.1	40.2	69.0	79.2
IMRAM	BUTD, Full	54.7	84.2	91.0	41.0	69.2	79.9

Performance of Flickr30K

Method_name	Concise_note	Sentence retrieval			Image retrieval
		R@1	R@5	R@10	R@1	R@5	R@10
DeViSE	RCNN	4.5	18.1	29.2	6.7	21.9	32.7
SDT-RNN	RCNN	9.6	29.8	41.1	8.9	29.8	41.1
DeFrag	RCNN	14.2	37.7	51.3	10.2	30.8	44.2
DeFrag	ftRCNN	16.4	40.2	54.7	10.3	31.4	44.5
DCCA	AlexNet	16.7	39.3	52.9	12.6	31.0	43.0
NIC	GoogleNet	17	--	56	17	--	57
DVSA	DepTree	20.0	46.6	59.4	15.0	36.5	48.2
DVSA	RCNN	22.2	48.2	61.4	15.2	37.7	50.5
UVSE	AlexNet	14.8	39.2	50.9	11.8	34.0	46.3
UVSE	VggNet	23.0	50.7	62.9	16.8	42.0	56.5
LRCN	VggNet	23.6	46.6	58.3	17.5	40.3	50.8
m-CNN*	OverFeat	20.1	44.2	56.3	15.9	40.3	51.9
m-CNN*	VggNet	33.6	64.1	74.9	26.2	56.3	69.6
m-RNN	AlexNet	18.4	40.2	50.9	12.6	31.2	41.5
m-RNN	VggNet	35.4	63.8	73.7	22.8	50.7	63.1
FV	GMM+HGLMM	35.0	62.0	73.8	25.0	52.7	66.0
HM-LSTM	RCNN	38.1	--	76.5	27.7	--	68.8
SPE	VggNet	40.3	68.9	79.9	29.7	60.1	72.1
sm-LSTM	VggNet	42.4	67.5	79.9	28.2	57.0	68.4
sm-LSTM*	VggNet	42.5	71.9	81.5	30.2	60.4	72.3
CSE	ResNet	44.6	74.3	83.8	36.9	69.1	79.6
MDM	VggNet	44.9	75.4	84.4	34.4	67.0	77.7
RRF-Net	ResNet	47.6	77.4	87.1	35.4	68.3	79.9
CMPL	MobileNet	40.3	66.9	76.7	30.4	58.2	68.5
CMPL	ResNet	49.6	76.8	86.1	37.3	65.7	75.5
2WayNet	VggNet	49.8	67.5	--	36.0	55.6	--
MFM	VggNet	50.2	78.1	86.7	38.2	70.1	80.2
VSE++	VggNet	41.3	69.1	77.9	31.4	60.0	71.2
VSE++	ResNet	52.9	80.5	87.2	39.6	70.1	79.5
TIMAM	ResNet, Bert	53.1	78.8	87.6	42.6	71.6	81.9
TERN	BUTD, Bert	53.2	79.4	86.0	41.1	71.9	81.2
DAN	VggNet	41.4	73.5	82.5	31.8	61.7	72.5
DAN	ResNet	55.0	81.8	89.0	39.4	69.2	79.1
NAA	ResNet	55.1	80.3	89.6	39.4	68.8	79.9
SCO	VggNet	44.2	74.1	83.6	32.8	64.3	74.9
SCO	ResNet	55.5	82.0	89.3	41.1	70.5	80.1
Dual-Path	VggNet	47.6	77.3	87.1	35.3	66.6	78.2
Dual-Path	ResNet	55.6	81.9	89.5	39.1	69.2	80.9
ITMeetsAL	VggNet	38.5	66.5	76.3	30.7	59.4	70.3
ITMeetsAL	MobileNet	46.6	73.5	82.5	34.4	63.3	74.2
ITMeetsAL	ResNet	56.5	82.2	89.6	43.5	71.8	80.2
CVSE++	ResNet	56.6	82.5	90.2	42.4	71.6	80.8
GXN	ResNet	56.8	--	89.6	41.5	--	80.1
SMAN	ResNet, Random	56.9	84.8	91.9	43.2	73.3	83.5
SMAN	ResNet, Glove	57.3	85.3	92.2	43.4	73.7	83.4
M3A	ResNet	58.1	82.8	90.1	44.7	72.4	81.1
Align2Ground	BUTD	--	--	--	49.7	74.8	83.3
A3VSE	BUTD	65.0	89.2	94.5	49.5	79.5	86.6
DXR	ResNet, Bert	65.1	87.3	92.6	50.6	78.8	86.7
MTFN	BUTD	63.1	85.8	92.4	46.3	75.3	83.6
MTFN	BUTD, RR_no_STT	65.3	88.3	93.3	46.7	75.9	83.8
MTFN	BUTD, RR_STT	65.3	88.3	93.3	52.0	80.1	86.1
R-SCAN	BUTD, VrR-VG	66.3	90.6	96.0	51.4	77.8	84.9
SAVE	ResNet	67.2	88.3	94.2	49.8	78.7	86.2
SCAN	BUTD, T2I_AVE	61.8	87.5	93.7	45.8	74.4	83.0
SCAN	BUTD, I2T_AVE	67.9	89.0	94.4	43.9	74.2	82.8
SCAN*	BUTD, AVE+LSE	67.4	90.3	95.8	48.6	77.7	85.2
BFAN	BUTD, prob	65.5	89.4	--	47.9	77.6	--
BFAN	BUTD, equal	64.5	89.7	--	48.8	77.3	--
BFAN*	BUTD	68.1	91.4	--	50.8	78.4	--
CAMP	BUTD	68.1	89.7	95.2	51.5	77.1	85.3
RDAN	BUTD	68.1	91.0	95.9	54.1	80.9	87.2
GSLS	ResNet, BUTD	68.2	89.1	94.5	43.4	73.5	82.5
Personality	ResNeXt, Transformer	68.4	90.6	95.3	--	--	--
CASC	ResNet	68.5	90.6	95.9	50.2	78.3	86.3
GVSE*	BUTD	68.5	90.9	95.5	50.6	79.8	87.6
HAL	SCAN_I2T	68.6	89.9	94.7	46.0	74.0	82.3
OAN	BUTD	68.6	93.0	96.0	53.3	80.1	87.1
SAEM	BUTD, Bert	69.1	91.0	95.1	52.4	81.1	88.1
MPL	SCAN_I2T	69.4	89.9	95.4	47.5	75.5	83.1
LIWE	BUTD, CLMR	64.0	88.3	93.3	46.8	76.4	84.5
LIWE	BUTD, -Glove	66.4	88.9	94.1	47.5	76.2	84.9
LIWE	BUTD, +Glove	69.6	90.3	95.6	51.2	80.4	87.2
PFAN	BUTD, T2I	66.0	89.6	94.3	49.6	77.0	84.2
PFAN	BUTD, I2T	67.6	90.0	93.8	45.7	74.7	83.6
PFAN*	BUTD	70.0	91.8	95.0	50.4	78.7	86.1
PFAN++*	BUTD	70.1	91.8	96.1	52.7	79.9	87.0
CAAN	BUTD	70.1	91.6	97.2	52.8	79.0	87.9
DP-RNN	BUTD	70.2	91.6	95.8	55.5	81.3	88.2
TERAN	BUTD, Bert	70.8	90.9	95.5	56.5	81.2	88.2
HOAD	BUTD	70.8	92.7	96.0	59.5	85.6	91.0
HOAD	BUTD, +Dist	70.8	92.7	96.0	60.9	86.1	91.0
GOT	SCAN_I2T	70.9	92.8	95.5	50.7	78.7	86.2
LGSGM	BUTD	71.0	91.9	96.1	57.4	84.1	90.2
VSRN	BUTD	70.4	89.2	93.7	53.0	77.9	85.7
VSRN*	BUTD	71.3	90.6	96.0	54.7	81.8	88.2
SCG	VggNet, Prod	57.2	85.1	92.1	40.1	69.5	79.5
SCG	VggNet, Gated	71.8	90.8	94.8	49.3	76.4	85.6
SGM	BUTD	71.8	91.7	95.5	53.5	79.6	86.5
Meta-SPN*	BFAN*	72.5	93.2	96.7	53.3	80.2	87.2
CSCC	BUTD, +GloVe	72.7	93.4	96.5	61.2	86.7	91.5
ADDR*	BUTD, BFAN	71.3	91.5	96.4	54.0	80.0	87.6
ADDR*	BUTD, SCAN	72.1	93.1	96.1	53.5	80.4	87.4
ADDR*	BUTD, VSRN	73.0	92.5	96.6	55.6	82.0	88.9
AOQ*	BUTD, SCAN	70.3	92.0	95.5	50.0	79.2	86.2
AOQ*	BUTD, VSRN	72.8	91.8	95.8	55.3	82.2	88.4
AOQ*	BUTD, BFAN	73.2	94.5	97.0	54.0	80.3	87.7
CVSE	BUTD	73.5	92.1	95.8	52.9	80.4	87.8
SMFEA	BUTD	73.7	92.5	96.1	54.7	82.1	88.4
IMRAM	BUTD, Image	67.0	90.5	95.6	51.2	78.2	85.5
IMRAM	BUTD, Text	68.8	91.6	96.0	53.0	79.0	87.1
IMRAM	BUTD, Full	74.1	93.0	96.6	53.9	79.4	87.2
HAN	BUTD	74.1	92.4	96.4	54.8	81.1	87.4
MMCA	BUTD, Bert	74.2	92.8	96.4	54.8	81.4	87.8
SHAN	BUTD, T2I	72.5	92.3	95.8	53.6	78.6	85.5
SHAN	BUTD, I2T	70.6	91.7	95.5	50.5	77.1	85.2
SHAN	BUTD, Full	74.6	93.5	96.9	55.3	81.3	88.4
WCGL	BUTD	74.8	93.3	96.8	54.8	80.6	87.5
CCRS*	BUTD, SCAN	70.1	92.0	96.0	52.3	79.9	86.8
CCRS*	BUTD, BFAN	75.3	93.6	96.7	55.4	81.3	87.7
SSAMT	BUTD, Bert	75.4	92.6	96.4	54.8	81.5	88.0
SAN^	VggNet	67.0	88.0	94.6	51.4	77.2	85.2
SAN^	ResNet	75.5	92.6	96.2	60.1	84.7	90.6
SAM	VSRN	68.4	89.7	94.8	52.4	78.7	86.6
SAM^	CVSE^	70.0	89.2	93.1	55.0	82.6	89.0
SAM	SGR	75.9	92.4	96.6	57.6	83.1	89.7
GSMN	BUTD, sparse	71.4	92.0	96.1	53.9	79.7	87.1
GSMN	BUTD, dense	72.6	93.5	96.8	53.7	80.0	87.0
GSMN*	BUTD	76.4	94.3	97.3	57.4	82.3	89.0
ADAPT	BUTD, I2T	70.2	90.8	95.8	55.5	82.7	89.8
ADAPT	BUTD, T2I	73.6	93.7	96.7	57.0	83.6	90.3
ADAPT*	BUTD, +GloVe	76.6	95.4	97.6	60.7	86.6	92.0
SGRAF	BUTD, SAF	73.7	93.3	96.3	56.1	81.5	88.0
SGRAF	BUTD, SGR	75.2	93.3	96.6	56.2	81.0	86.5
SGRAF*	BUTD	77.8	94.1	97.4	58.5	83.0	88.8
DSRAN	BUTD, GRU	72.6	93.6	96.3	56.3	84.0	89.8
DSRAN	BUTD, Bert	75.3	94.4	97.6	57.3	84.8	90.9
DSRAN*	BUTD, GRU	74.9	94.5	97.0	58.6	85.8	91.3
DSRAN*	BUTD, Bert	77.8	95.1	97.6	59.2	86.0	91.9
CAMERA	BUTD, Bert	76.5	95.1	97.2	58.9	84.7	90.2
CAMERA*	BUTD, Bert	78.0	95.1	97.9	60.3	85.9	91.7
CAEMCL	BUTD	76.3	93.2	96.5	57.0	82.1	88.5
CAEMCL*	BUTD	78.7	94.5	97.9	58.2	83.6	89.6
T-EMDE	BUTD, SAF	75.2	94.2	97.1	57.1	82.2	88.3
T-EMDE	BUTD, SGR	77.5	93.1	97.2	56.9	82.0	87.5
T-EMDE*	BUTD, SGRAF	78.8	94.4	97.5	59.6	83.6	89.2
DIME	BUTD, I2T, Bert	77.4	95.0	97.4	60.1	85.5	91.8
DIME	BUTD, T2I, Bert	77.5	93.5	97.5	59.1	85.5	91.0
DIME*	BUTD, Bert	81.0	95.9	98.4	63.6	88.1	93.0
PG*	BUTD, +3loss	81.0	94.5	97.1	60.6	86.5	92.4
PG*	BUTD, +GloVe	82.8	95.9	97.9	62.2	89.3	93.8
ACMM	BUTD	80.0	95.5	98.2	50.2	76.8	84.7
ACMM*	BUTD	85.2	96.7	98.4	53.8	79.8	86.8
GPO	IN, BiGRU	77.1	94.5	97.1	58.5	84.1	89.6
GPO*	IN+VG, BiGRU	80.7	96.4	98.3	60.8	86.3	92.3
GPO*	IN+VG, Bert	85.3	97.2	98.9	66.7	89.9	94.0
GPO*	WSL, Bert	88.7	98.9	99.8	76.1	94.5	97.1

Performance of MSCOCO1K

Method_name	Concise_note	Sentence retrieval			Image retrieval
		R@1	R@5	R@10	R@1	R@5	R@10
STV	combine-skip	33.8	67.7	82.1	25.9	60.0	74.6
DVSA	RCNN	38.4	69.9	80.5	27.4	60.2	74.8
FV	GMM+HGLMM	39.4	67.9	80.9	25.1	59.8	76.6
m-RNN	VggNet	41.0	73.0	83.5	29.0	42.2	77.0
m-CNN*	VggNet	42.8	73.1	84.1	32.6	68.6	82.8
UVSE	VggNet	43.4	75.7	85.8	31.0	66.7	79.9
HM-LSTM	RCNN	43.9	--	87.8	36.1	--	86.7
Order-emb	VggNet	46.7	--	88.9	37.9	--	85.9
SPE	VggNet	50.1	79.7	89.2	39.6	75.2	86.9
SEAM	VggNet	50.7	81.4	90.9	40.3	75.7	87.4
sm-LSTM	VggNet	52.4	81.7	90.8	38.6	73.4	84.6
sm-LSTM*	VggNet	53.2	83.1	91.5	40.7	75.8	87.4
CMPL	MobileNet	52.9	83.8	92.1	41.3	74.6	85.9
MDM	VggNet	54.7	84.1	91.9	44.6	79.6	90.5
2WayNet	VggNet	55.8	75.2	--	39.7	63.3	--
CMPM	ResNet	56.1	86.3	92.9	44.6	78.8	89.0
CSE	ResNet	56.3	84.4	92.2	45.7	81.2	90.6
RRF-Net	ResNet	56.4	85.3	91.5	43.9	78.1	88.6
ITMeetsAL	VggNet	44.2	76.1	86.3	37.1	72.7	85.1
ITMeetsAL	MobileNet	54.7	84.3	91.1	41.0	76.7	88.1
ITMeetsAL	ResNet	58.5	85.3	92.1	48.3	82.0	90.6
MFM	VggNet	58.9	86.3	92.4	47.7	81.0	90.9
CHAIN-VSE	VggNet	51.6	82.0	91.3	38.6	75.1	87.2
CHAIN-VSE	ResNet	59.4	88.0	94.2	43.5	79.8	90.2
NAA	ResNet	61.3	87.9	95.4	47.0	80.8	90.1
TERN	BUTD, Bert	63.7	90.5	96.2	51.9	85.6	93.6
VSE++	VggNet	57.2	86.0	93.3	45.9	79.4	89.1
VSE++	ResNet	64.6	90.0	95.7	52.0	84.3	92.0
Dual-Path	VggNet	59.4	86.2	92.9	41.6	76.3	87.5
Dual-Path	ResNet	65.6	89.8	95.5	47.1	79.9	90.0
DXR	ResNet, Bert	67.0	93.0	97.6	56.8	88.2	94.9
Personality	ResNeXt, Transformer	67.3	91.7	96.5	--	--	--
Align2Ground	BUTD	--	--	--	56.6	84.9	92.8
SMAN	ResNet, Random	67.9	90.6	96.2	58.8	87.0	93.7
SMAN	ResNet, Glove	68.4	91.3	96.6	58.5	87.4	93.5
GXN	ResNet	68.5	--	97.9	56.6	--	94.5
GSLS	ResNet, BUTD	68.9	94.1	98.0	58.6	88.2	94.9
CVSE++	ResNet	69.1	92.2	96.1	55.6	86.7	93.8
PVSE	ResNet	69.2	91.6	96.6	55.2	86.5	93.7
DSVE-Loc	ResNet	69.8	91.9	96.6	55.9	86.9	94.0
SCO	VggNet	66.6	91.8	96.6	55.5	86.6	93.8
SCO	ResNet	69.9	92.9	97.5	56.7	87.5	94.8
R-SCAN	BUTD, VrR-VG	70.3	94.5	98.1	57.6	87.3	93.7
M3A	ResNet	70.4	91.7	96.8	58.4	87.1	94.0
SAVE	ResNet	70.8	93.2	97.6	56.9	87.6	94.4
MPL	SCAN_I2T	71.1	93.7	98.2	56.8	86.7	93.0
SAEM	BUTD, Bert	71.2	94.1	97.7	57.8	88.6	94.9
SoDeep	DSVE-Loc	71.5	92.8	97.1	56.2	87.0	94.3
OAN	BUTD	71.7	96.4	99.3	60.2	88.6	94.5
GVSE*	BUTD	72.2	94.1	98.1	60.5	89.4	95.8
CAMP	BUTD	72.3	94.8	98.3	58.5	87.9	95.0
CASC	ResNet	72.3	96.0	99.0	58.9	89.8	96.0
SCAN	BUTD, T2I_AVE	70.9	94.5	97.8	56.4	87.0	93.9
SCAN	BUTD, I2T_AVE	69.2	93.2	97.5	54.4	86.0	93.6
SCAN*	BUTD, LSE+AVE	72.7	94.8	98.4	58.8	88.4	94.8
LIWE	BUTD, -Glove	69.6	93.9	98.0	55.5	87.3	94.2
LIWE	BUTD, CLMR	71.8	93.1	97.6	56.2	87.5	94.2
LIWE	BUTD, +Glove	73.2	95.5	98.2	57.9	88.3	94.5
SGM	BUTD	73.4	93.8	97.8	57.5	87.3	94.3
ParNet	BUTD, NP	72.8	94.9	97.9	57.9	87.4	94.0
ParNet	BUTD, P	73.5	94.5	98.3	58.3	88.2	94.1
MTFN	BUTD	71.9	94.2	97.9	57.3	88.6	95.0
MTFN	BUTD, RR_no_STT	74.3	94.9	97.9	57.5	88.8	95.0
MTFN	BUTD, RR_STT	74.3	94.9	97.9	60.1	89.1	95.0
Meta-SPN	BFAN, equal	74.4	95.0	98.3	58.6	87.6	94.3
RDAN	BUTD	74.6	96.2	98.7	61.6	89.2	94.7
CVSE	BUTD	74.8	95.1	98.3	59.9	89.4	95.2
MMCA	BUTD, Bert	74.8	95.6	97.7	61.6	89.8	95.2
BFAN	BUTD, prob	73.0	94.8	--	58.0	87.6	--
BFAN	BUTD, equal	73.7	94.9	--	58.3	87.5	--
BFAN*	BUTD	74.9	95.2	--	59.4	88.4	--
SMFEA	BUTD	75.1	95.4	98.3	62.5	90.1	96.2
DP-RNN	BUTD	75.3	95.8	98.6	62.5	89.7	95.1
CCRS*	BUTD, SCAN	70.9	94.3	98.0	57.3	87.6	94.3
CCRS*	BUTD, BFAN	75.4	95.3	98.5	60.3	88.6	94.6
WCGL	BUTD	75.4	95.5	98.6	60.8	89.3	95.3
CAAN	BUTD	75.5	95.4	98.5	61.3	89.7	95.2
VSRN	BUTD	74.0	94.3	97.8	60.8	88.4	94.1
VSRN*	BUTD	76.2	94.8	98.2	62.8	89.7	95.1
ADAPT	BUTD, I2T	74.5	94.2	97.9	62.0	90.4	95.5
ADAPT	BUTD, T2I	75.3	95.1	98.4	63.3	90.0	95.5
ADAPT*	BUTD	76.5	95.6	98.9	62.2	90.5	96.0
PFAN	BUTD, T2I	75.8	95.9	99.0	61.0	89.1	95.1
PFAN	BUTD, I2T	70.7	94.1	97.8	53.0	84.5	92.6
PFAN*	BUTD	76.5	96.3	99.0	61.6	89.6	95.2
SCG	VggNet, Prod	73.4	94.8	97.6	56.3	85.6	93.5
SCG	VggNet, Gated	76.6	96.3	99.2	61.4	88.9	95.1
IMRAM	BUTD, Image	76.1	95.3	98.2	61.0	88.6	94.5
IMRAM	BUTD, Text	74.0	95.6	98.4	60.6	88.9	94.6
IMRAM	BUTD, Full	76.7	95.6	98.5	61.7	89.1	95.0
SHAN	BUTD, T2I	75.9	96.1	98.7	60.7	88.2	94.2
SHAN	BUTD, I2T	73.0	95.8	97.9	58.5	87.3	94.0
SHAN	BUTD, Full	76.8	96.3	98.7	62.6	89.6	95.8
PFAN++*	BUTD	77.1	96.5	98.3	62.5	89.9	95.4
ADDR*	BUTD, SCAN	76.1	95.5	98.4	61.2	88.9	94.8
ADDR*	BUTD, BFAN	76.4	95.8	98.3	62.3	89.4	96.2
ADDR*	BUTD, VSRN	77.4	96.1	98.9	63.5	90.7	96.7
CAMERA	BUTD, Bert	75.9	95.5	98.6	62.3	90.1	95.2
CAMERA*	BUTD, Bert	77.5	96.3	98.8	63.4	90.9	95.8
AOQ*	BUTD, SCAN	74.1	95.2	98.5	59.8	88.6	95.0
AOQ*	BUTD, BFAN	77.3	96.0	98.5	61.2	89.2	95.0
AOQ*	BUTD, VSRN	77.5	95.5	98.6	63.5	90.5	95.8
TERAN	BUTD, Bert	77.7	95.9	98.6	65.0	91.2	96.4
HOAD^	BUTD	77.0	96.1	98.7	65.1	93.1	97.9
HOAD^	BUTD, +Dist	77.8	96.1	98.7	66.2	93.0	97.9
TOD-Net	VSE++	68.6	92.0	96.9	54.5	85.3	92.4
TOD-Net	Bert	75.8	95.3	98.4	61.8	89.6	95.0
TOD-Net*	Bert	78.1	96.0	98.6	63.6	90.6	95.8
SSAMT	BUTD, Bert	78.2	95.6	98.0	62.7	89.6	95.3
HAL	SCAN_I2T	78.3	96.3	98.5	60.1	86.7	92.8
DSRAN	BUTD, GRU	76.3	94.9	98.4	62.4	89.7	95.2
DSRAN	BUTD, Bert	77.1	95.3	98.1	62.9	89.9	95.3
DSRAN*	BUTD, GRU	78.0	95.6	98.5	64.2	90.4	95.8
DSRAN*	BUTD, Bert	78.3	95.7	98.4	64.5	90.8	95.8
GSMN	BUTD, sparse	76.1	95.6	98.3	60.4	88.7	95.0
GSMN	BUTD, dense	74.7	95.3	98.2	60.3	88.5	94.6
GSMN*	BUTD	78.4	96.4	98.6	63.3	90.1	95.7
HAN	BUTD	78.7	96.4	98.8	65.4	90.5	95.3
DIME	BUTD, I2T, Bert	77.9	95.9	98.3	63.0	90.5	96.2
DIME	BUTD, T2I, Bert	77.2	95.5	98.5	62.3	90.2	95.8
DIME*	BUTD, Bert	78.8	96.3	98.7	64.8	91.5	96.5
CSCC^	BUTD, +GloVe	78.8	96.1	99.0	66.6	92.5	96.4
CAEMCL	BUTD	77.6	96.4	98.8	62.2	89.8	95.8
CAEMCL*	BUTD	78.9	97.5	98.8	65.7	90.2	96.6
SGRAF	BUTD, SAF	76.1	95.4	98.3	61.8	89.4	95.3
SGRAF	BUTD, SGR	78.0	95.8	98.2	61.4	89.3	95.4
SGRAF*	BUTD	79.6	96.2	98.5	63.2	90.7	96.1
T-EMDE	BUTD, SAF	78.3	95.7	98.5	62.3	89.7	95.2
T-EMDE	BUTD, SGR	77.1	95.9	98.5	61.6	89.5	95.1
T-EMDE*	BUTD, SGRAF	79.6	96.3	98.7	63.5	90.4	95.6
SAM	VSRN	74.6	93.6	97.5	61.5	89.6	94.9
SAM^	CVSE^	79.8	95.1	97.7	67.0	93.0	97.3
SAM	SGR	80.7	97.2	98.6	63.8	90.5	95.9
ACMM	BUTD	81.9	98.0	99.3	58.2	87.3	93.9
ACMM*	BUTD	84.1	97.8	99.4	60.7	88.7	94.9
PG*	BUTD, +GloVe	84.0	95.8	97.8	63.9	88.9	95.6
SAN^	VggNet	74.9	94.9	98.2	60.8	90.3	95.7
SAN^	ResNet	85.4	97.5	99.0	69.1	93.4	97.2
GPO	IN, BiGRU	76.5	95.3	98.5	62.9	90.6	95.8
GPO*	IN+VG, BiGRU	80.0	97.0	99.0	64.8	91.6	96.5
GPO*	IN+VG, Bert	82.2	97.5	99.5	68.1	92.9	97.2
GPO*	WSL, Bert	85.6	98.0	99.4	73.1	94.3	97.7

Performance of MSCOCO5K

Method_name	Concise_note	Sentence retrieval			Image retrieval
		R@1	R@5	R@10	R@1	R@5	R@10
DVSA	RCNN	16.5	39.2	52.0	10.7	29.6	42.2
FV	GMM+HGLMM	17.3	39.0	50.2	10.8	28.3	40.1
Order-emb	VggNet	23.3	--	65.0	18.0	--	57.6
CSE	ResNet	27.9	57.1	70.4	22.2	50.2	64.4
CMPL	MobileNet	24.6	52.3	66.4	19.1	44.6	58.4
CMPM	ResNet	31.1	60.7	73.9	22.9	50.2	63.8
TERN	BUTD, Bert	38.4	69.5	81.3	28.7	59.7	72.7
Dual-Path	VggNet	35.5	63.2	75.6	21.0	47.5	60.9
Dual-Path	ResNet	41.2	70.5	81.1	25.3	53.4	66.4
VSE++	VggNet	32.9	61.7	74.7	24.1	52.8	66.2
VSE++	ResNet	41.3	71.1	81.2	30.3	59.4	72.4
GXN	ResNet	42.0	--	84.7	31.7	--	74.6
SCO	VggNet	40.2	70.1	81.3	31.3	61.5	73.9
SCO	ResNet	42.8	72.3	83.0	33.1	62.9	75.5
CVSE++	ResNet	43.2	73.5	84.1	32.4	62.2	74.6
DXR	ResNet, Bert	44.9	75.2	84.7	33.9	64.9	77.4
PVSE	ResNet	45.2	74.3	84.5	32.4	63.0	75.0
R-SCAN	BUTD, VrR-VG	45.4	77.9	87.9	36.2	65.5	76.7
SAVE	ResNet	46.7	76.3	86.1	34.0	64.8	77.0
MPL	SCAN_I2T	46.9	77.7	87.6	34.4	64.2	75.9
GVSE*	BUTD	47.2	76.6	88.4	31.2	61.2	70.5
CASC	ResNet	47.2	78.3	87.4	34.7	64.8	76.8
OAN	BUTD	47.8	81.2	90.4	37.0	66.6	78.0
MTFN	BUTD	44.7	76.4	87.3	33.1	64.7	76.1
MTFN	BUTD, RR	48.3	77.6	87.3	35.9	66.1	76.1
M3A	ResNet	48.9	75.2	84.4	38.3	65.7	76.9
A3VSE	BUTD	49.3	81.1	90.2	39.0	68.0	80.1
GVSE*	BUTD	49.9	77.4	87.6	38.4	68.5	79.7
SGM	BUTD	50.0	79.3	87.9	35.3	64.9	76.5
CAMP	BUTD	50.1	82.1	89.7	39.0	68.9	80.2
SCAN	BUTD, I2T_LSE	46.4	77.4	87.2	34.4	63.7	75.7
SCAN*	BUTD, AVE+LSE	50.4	82.2	90.0	38.6	69.3	80.4
GOT	SCAN_I2T	50.5	80.2	89.8	38.1	66.8	78.5
PFAN*	BUTD	50.8	83.9	89.1	39.5	69.5	80.8
Meta-SPN	BFAN, equal	51.0	81.1	89.4	37.5	66.7	77.5
PFAN++*	BUTD	51.2	84.3	89.2	41.4	70.9	79.0
HOAD	BUTD	51.2	81.7	89.1	39.4	72.5	84.1
HOAD	BUTD, +Dist	51.4	81.8	89.1	40.5	73.5	84.1
CAAN	BUTD	52.5	83.3	90.9	41.2	70.3	82.9
VSRN*	BUTD	53.0	81.1	89.4	40.5	70.6	81.1
CCRS*	BUTD, SCAN	47.9	78.1	88.2	36.9	66.9	78.4
CCRS*	BUTD, BFAN	53.1	81.8	90.2	38.3	67.8	78.6
IMRAM	BUTD, Image	53.2	82.5	90.4	38.9	68.5	79.2
IMRAM	BUTD, Text	52.0	81.8	90.1	38.6	68.1	79.1
IMRAM	BUTD, Full	53.7	83.2	91.0	39.7	69.1	79.8
MMCA	BUTD, Bert	54.0	82.5	90.7	38.7	69.7	80.8
SMFEA	BUTD	54.2	--	89.9	41.9	--	83.7
CAMERA	BUTD, Bert	53.1	81.3	89.8	39.0	70.5	81.5
CAMERA*	BUTD, Bert	55.1	82.9	91.2	40.5	71.7	82.5
DSRAN	BUTD, GRU	51.9	81.6	89.8	39.5	70.6	81.0
DSRAN	BUTD, Bert	53.7	82.1	89.9	40.3	70.9	81.3
DSRAN*	BUTD, GRU	54.4	83.5	91.3	41.5	71.9	82.1
DSRAN*	BUTD, Bert	55.3	83.5	90.9	41.7	72.7	82.8
CSCC	BUTD, +GloVe	55.6	83.6	91.2	40.8	73.2	84.3
TERAN	BUTD, Bert	55.6	83.9	91.6	42.6	72.5	82.9
SAM	VSRN	49.1	79.0	87.4	37.5	68.1	79.5
SAM	SGR	55.7	83.2	91.2	40.5	69.7	80.5
SAM^	CVSE^	56.4	82.4	90.1	42.3	73.9	84.5
SCG	VggNet, Prod	49.9	78.9	88.1	33.2	62.4	74.7
SCG	VggNet, Gated	56.6	84.5	92.0	39.2	68.0	81.3
AOQ*	BUTD, SCAN	51.2	82.5	90.1	39.4	69.7	80.4
AOQ*	BUTD, VSRN	55.1	83.3	90.8	41.1	71.5	82.0
AOQ*	BUTD, BFAN	57.3	84.5	91.7	40.1	69.2	80.1
ADDR*	BUTD, BFAN	54.3	84.0	91.5	40.1	69.2	80.6
ADDR*	BUTD, VSRN	56.6	85.3	90.4	42.5	71.9	82.0
ADDR*	BUTD, SCAN	57.3	86.0	92.7	41.8	72.0	81.3
SSAMT	BUTD, Bert	57.7	84.2	90.8	40.8	70.5	80.5
SGRAF	BUTD, SAF	53.3	82.3	90.1	39.8	69.0	80.2
SGRAF	BUTD, SGR	56.9	83.2	90.5	40.2	69.0	79.8
SGRAF*	BUTD	57.8	84.9	91.6	41.9	70.7	81.3
T-EMDE	BUTD, SAF	56.7	--	90.7	40.3	--	80.4
T-EMDE	BUTD, SGR	57.0	--	91.0	40.0	--	80.1
T-EMDE*	BUTD, SGRAF	59.1	--	91.8	41.8	--	81.7
DIME	BUTD, I2T, Bert	56.1	83.2	91.1	40.2	70.7	81.4
DIME	BUTD, T2I, Bert	55.3	82.4	90.2	39.7	70.3	81.0
DIME*	BUTD, Bert	59.3	85.4	91.9	43.1	73.0	83.1
SAN^	ResNet	65.4	89.4	94.8	46.2	77.4	86.6
ACMM	BUTD	63.5	88.0	93.6	36.7	65.1	76.7
ACMM*	BUTD	66.9	89.6	94.9	39.5	69.6	81.1
GPO	IN, BiGRU	55.1	81.9	89.9	40.9	70.6	81.5
GPO*	IN+VG, BiGRU	59.8	86.1	92.8	42.7	72.8	83.3
GPO*	IN+VG, Bert	62.5	87.8	94.0	46.0	75.8	85.7
GPO*	WSL, Bert	68.1	90.2	95.2	52.7	80.2	88.3
PG*	BUTD, +GloVe	68.7	88.7	93.0	46.2	77.8	85.5

Peformance of Identity-aware Datasets

Performance of RSTPReid

Method_name	Concise_note	Text-to-Image
		R@1	R@5	R@10
DSSL	ResNet	32.43	55.08	63.19

Performance of CUHK-PEDES

Method_name	Concise_note	Text-to-Image
		R@1	R@5	R@10
LSTM-Q+I	VggNet	17.19	--	57.82
GNA-RNN	VggNet	19.05	--	53.64
IATV	VggNet	25.94	--	60.48
PWM-ATH	VggNet	27.14	49.45	61.02
GLA	ResNet	43.58	66.93	76.26
Dual-Path	VggNet	32.15	54.42	64.30
Dual-Path	ResNet	44.40	66.26	75.07
CMPM	MobileNet	44.02	--	77.00
CMPL	MobileNet	49.37	--	79.27
MCCL	MobileNet, CL	48.21	--	78.27
MCCL	MobileNet	50.58	--	79.06
MIA	VggNet	48.00	70.70	79.30
MIA	ResNet	53.10	75.00	82.90
A-GANet	ResNet	53.14	74.03	82.95
PMA	VggNet	47.02	68.54	78.06
PMA	ResNet	53.81	73.54	81.23
TIMAM	ResNet, Bert	54.51	77.56	84.78
CMAAM	MobileNet	55.13	76.14	83.77
ITMeetsAL	MobileNet	51.85	73.36	81.27
ITMeetsAL	ResNet	55.72	76.15	84.26
ViTAA	ResNet	55.97	75.84	83.52
FTD	ResNet	57.84	78.33	85.43
MGEL	VggNet	52.68	74.37	83.11
MGEL	MobileNet	59.21	79.16	85.88
MGEL	ResNet	60.27	80.01	86.74
SSAN	VggNet	55.52	76.17	83.45
SSAN	ResNet	61.37	80.15	86.73
NAFS	ResNet, Bert	59.94	79.86	86.70
NAFS	+RVN	61.50	81.19	87.51
DSSL	ResNet	59.98	80.41	87.56
DSSL	+RR	62.33	82.11	88.01
LapsCore	CMPL	53.33	--	83.20
LapsCore	NAFS	63.40	--	87.80

Performance of ICFG-PEDES

Method_name	Concise_note	Text-to-Image
		R@1	R@5	R@10
Dual-Path	ResNet	38.99	59.44	68.41
CMPL	ResNet	43.51	65.44	74.26
MIA	ResNet	46.49	67.14	75.18
SCAN	ResNet	50.05	69.65	77.21
ViTAA	ResNet	50.98	68.79	75.78
SSAN	ResNet	54.23	72.63	79.53

Performance of CUB-Flowers

Method_name	Concise_note	CUB		Flowers
		Image-to-Text	Text-to-Image	Image-to-Text	Text-to-Image
		R@1	AP@50	R@1	AP@50
FV	GMM+HGLMM	36.5	35.6	54.8	52.8
Word2Vec		38.6	33.5	54.2	52.1
Word-NN	CNN	51.0	43.3	60.7	56.3
Word-NN	CNN-RNN	56.8	48.7	65.6	59.6
IATV	Triplet	52.5	52.4	64.3	64.9
IATV	VggNet	61.5	57.6	68.4	70.1
CMPM	MobileNet	62.1	64.6	66.1	67.7
CMPL	MobileNet	64.3	67.9	68.9	69.7
TIMAM	ResNet, Bert	67.7	70.3	70.6	73.7
LapsCore	CMPL	68.0	66.0	75.2	71.4
LapsCore	CMP_adv	72.3	69.5	77.9	73.3