VideoMind on Public Benchmarks

June 25, 2025 · View on GitHub

Grounded Video Question-Answering (Grounded VideoQA)
- CG-Bench
- ReXTime
- NExT-GQA
- DeVE-QA
Video Temporal Grounding (VTG)
Reasoning Temporal Localization (RTL)
- ActivityNet-RTL
General Video Question-Answering (VideoQA)
- Video-MME
- MLVU
- LVBench
- MVBench
- LongVideoBench

Click on the datasets to jump to the tables.

Method	Size	long-acc.	mIoU	rec.@IoU	acc.@IoU
GPT-4o	–	45.2	5.62	8.30	4.38
GPT-4o-mini	–	33.4	3.75	5.18	2.21
Gemini-1.5-Pro	–	37.2	3.95	5.81	2.53
Gemini-1.5-Flash	–	32.3	3.67	5.44	2.45
Claude-3.5-Sonnet	–	40.5	3.99	5.67	2.79
Video-LLaVA	7B	16.2	1.13	1.96	0.59
VideoLLaMA	7B	18.4	1.21	1.87	0.84
VideoChat2	7B	19.3	1.28	1.98	0.94
Qwen-VL-Chat	7B	21.6	0.89	1.19	0.42
ST-LLM	7B	23.8	2.23	2.86	1.13
ShareGPT4Video	16B	26.7	1.85	2.65	1.01
Chat-UniVi-v1.5	13B	25.9	2.07	2.53	1.21
VILA	8B	28.7	1.56	2.89	1.35
MiniCPM-v2.6	8B	30.1	2.35	2.61	1.04
LongVA	7B	28.7	2.94	3.86	1.78
LLaVA-OV	7B	31.1	1.63	1.78	1.08
Video-CCAM	14B	29.7	2.63	2.48	1.83
Kangaroo	8B	30.2	2.56	2.81	1.94
VITA	8×7B	33.3	3.06	3.53	2.06
Qwen2-VL	72B	41.3	3.58	5.32	3.31
InternVL2	78B	42.2	3.91	5.05	2.64
VideoMind (Ours)	2B	31.0	5.94	8.50	4.02
VideoMind (Ours)	7B	38.4	7.10	9.93	4.67

Method	Size	FT	R@0.3	R@0.5	mIoU	Acc	Acc@IoU
VTimeLLM	7B	×	28.84	17.41	20.14	36.16	–
TimeChat	7B	×	14.42	7.61	11.65	40.04	–
LITA	13B	×	29.49	16.29	21.49	34.44	–
VTimeLLM	7B	✓	43.69	26.13	29.92	57.58	17.13
TimeChat	7B	✓	40.13	21.42	26.29	49.46	10.92
VideoMind (Ours)	2B	×	34.31	22.69	24.83	69.06	17.26
VideoMind (Ours)	7B	×	38.22	25.52	27.61	74.59	20.20

Note: Acc@IoU means both QA (Acc) and Grounding (IoU >= 0.5) are correct

Model	Size	R@IoU-0.3	R@IoU-0.5	mIoU	R@IoP-0.3	R@IoP-0.5	mIoP	Acc@GQA
FrozenBiLM NG+	890M	13.5	6.1	9.6	28.5	23.7	24.2	17.5
VIOLETv2	–	4.3	1.3	3.1	25.1	23.3	23.6	12.8
SeViLA	4B	29.2	13.8	21.7	34.7	22.9	29.5	16.6
LangRepo	8×7B	–	12.2	18.5	–	28.7	31.3	17.1
VideoStreaming	8.3B	–	13.3	19.3	31.0	32.2	17.8	–
LLoVi	1.8T	–	15.3	20.0	–	36.9	37.3	24.3
HawkEye	7B	37.0	19.5	25.7	–	–	–	–
VideoChat-TPO	7B	41.2	23.4	27.7	47.5	32.8	35.6	25.5
VideoMind (Ours)	2B	45.2	23.2	28.6	51.3	32.6	36.4	25.2
VideoMind (Ours)	7B	50.2	25.8	31.4	56.0	35.3	39.0	28.2

Note: Acc@GQA means both QA (Acc) and Grounding (IoP >= 0.5) are correct

Model	Size	R@IoU-0.5	mIoU	R@IoP-0.5	mIoP	Acc@QA	Acc@GQA
VideoMind (Ours)	2B	21.7	26.3	50.7	49.9	76.5	41.2
VideoMind (Ours)	7B	26.5	30.1	52.3	51.9	81.0	44.2

Note: Acc@GQA means both QA (Acc) and Grounding (IoP >= 0.5) are correct

Method	Size	FT	R@0.3	R@0.5	R@0.7	mIoU
Moment-DETR	–	✓	65.8	52.1	30.6	45.5
UMT	–	✓	–	48.3	29.3	–
UniVTG	–	✓	70.8	58.1	35.6	50.1
R2-Tuning	–	✓	70.9	59.8	37.0	50.9
VTimeLLM	13B	×	55.3	34.3	14.7	34.6
TimeChat	7B	×	51.5	32.2	13.4	–
Momentor	7B	×	42.6	26.6	11.6	28.5
HawkEye	7B	×	50.6	31.4	14.5	33.7
ChatVTG	7B	×	52.7	33.0	15.9	34.9
VideoChat-TPO	7B	×	58.3	40.2	18.4	38.1
E.T. Chat	4B	×	65.7	45.9	20.0	42.3
VideoMind (Ours)	2B	×	67.6	51.1	26.0	45.2
VideoMind (Ours)	7B	×	73.5	59.1	31.2	50.2

Method	Size	FT	R@0.3	R@0.5	R@0.7	mIoU
2D-TAN	–	✓	60.4	43.4	25.0	42.5
MMN	–	✓	64.5	48.2	29.4	46.6
VDI	–	✓	48.1	28.8	–	–
VideoChat	7B	×	8.8	3.7	1.5	7.2
Video-LLaMA	7B	×	6.9	2.1	0.8	6.5
Video-ChatGPT	7B	×	26.4	13.6	6.1	18.9
Valley	7B	×	30.6	13.7	8.1	21.9
ChatVTG	7B	×	40.7	22.5	9.4	27.2
Momentor	7B	×	42.9	23.0	12.4	29.3
E.T. Chat	4B	×	24.1	12.8	6.1	18.9
VideoMind (Ours)	2B	×	44.0	26.5	12.6	30.1
VideoMind (Ours)	7B	×	48.4	30.3	15.7	33.3

Method	Size	FT	R1@0.5	R1@0.7	mAP@0.5	mAP@0.75	mAP Avg.
XML	–	✓	41.83	30.35	44.63	31.73	32.14
XML+	–	✓	46.69	33.46	47.89	34.67	34.90
Moment-DETR	–	✓	59.78	40.33	60.51	35.36	36.14
UMT	–	✓	60.83	43.26	57.33	39.12	38.08
MomentDiff	–	✓	58.21	41.48	54.57	37.21	36.84
QD-DETR	–	✓	62.40	44.98	62.52	39.88	39.86
UniVTG	–	✓	65.43	50.06	64.06	45.02	43.63
R2-Tuning	–	✓	68.03	49.35	69.04	47.56	46.17
VideoMind (Ours)	2B	✓	75.42	59.35	74.11	55.15	51.60
VideoMind (Ours)	7B	✓	78.53	61.09	76.07	58.17	54.19

Method	Size	FT	R@0.3	R@0.5	R@0.7	mIoU
2D-TAN	–	✓	40.0	28.0	12.9	27.2
VSLNet	–	✓	35.5	23.5	13.1	25.0
Moment-DETR	–	✓	38.0	24.7	12.0	25.5
UniVTG	–	✓	51.4	35.0	17.4	33.6
R2-Tuning	–	✓	49.7	38.7	25.1	35.9
VideoMind (Ours)	2B	✓	38.6	26.9	15.5	27.4
VideoMind (Ours)	7B	✓	49.5	36.2	21.4	34.4

Method	Size	FT	R@0.3	R@0.5	R@0.7	mIoU
2D-TAN	–	✓	4.3	1.8	0.6	3.4
VSLNet	–	✓	4.5	2.4	1.0	3.5
Moment-DETR	–	✓	4.3	1.8	0.7	3.5
UniVTG	–	✓	7.3	4.0	1.3	4.9
R2-Tuning	–	✓	7.2	4.5	2.1	4.9
UniVTG	–	×	6.5	3.5	1.2	4.6
VideoMind (Ours)	2B	×	5.9	2.9	1.2	4.7
VideoMind (Ours)	7B	×	7.2	3.7	1.7	5.4

Method	Size	FT	P@0.5	mIoU
LITA	7B	✓	21.2	24.1
LITA	13B	✓	25.9	28.6
VideoMind (Ours)	2B	×	20.1	22.7
VideoMind (Ours)	7B	×	28.0	31.3

Model	Size	All	Long
Gemini-1.5-Pro	–	75.0	67.4
GPT-4o	–	71.9	65.3
Video-LLaVA	7B	41.1	37.8
TimeChat	7B	34.3	32.1
MovieChat	7B	38.2	33.4
PLLaVA	34B	40.0	34.7
VideoChat-TPO	7B	48.8	41.0
LongVA	7B	52.6	46.2
VideoMind (Ours)	2B	53.6	45.4
VideoMind (Ours)	7B	58.2	49.2

Model	Size	AS	AP	AA	FA	UA	OE	OI	OS	MD	AL	ST	AC	MC	MA	SC	FP	CO	EN	ER	CI	Avg.
GPT-4V	–	55.5	63.5	72.0	46.5	73.5	18.5	59.0	29.5	12.0	40.5	83.5	39.0	12.0	22.5	45.0	47.5	52.0	31.0	59.0	11.0	43.5
Video-ChatGPT	7B	23.5	26.0	62.0	22.5	26.5	54.0	28.0	40.0	23.0	20.0	31.0	30.5	25.5	39.5	48.5	29.0	33.0	29.5	26.0	35.5	32.7
Video-LLaMA	7B	27.5	25.5	51.0	29.0	39.0	48.0	40.5	38.0	22.5	22.5	43.0	34.0	22.5	32.5	45.5	32.5	40.0	30.0	21.0	37.0	34.1
VideoChat	7B	33.5	26.5	56.0	33.5	40.5	53.0	40.5	30.0	25.5	27.0	48.5	35.0	20.5	42.5	46.0	26.5	41.0	23.5	23.5	36.0	35.5
Video-LLaVA	7B	46.0	42.5	56.5	39.0	53.5	53.0	48.0	41.0	29.0	31.5	82.5	45.0	26.0	53.0	41.5	33.5	41.5	27.5	38.5	31.5	43.0
TimeChat	7B	40.5	36.0	61.0	32.5	53.0	53.5	41.5	29.0	19.5	26.5	66.5	34.0	20.0	43.5	42.0	36.5	36.0	29.0	35.0	35.0	38.5
PLLaVA	7B	58.0	49.0	55.5	41.0	61.0	56.0	61.0	36.0	23.5	26.0	82.0	39.5	42.0	52.0	45.0	42.0	53.5	30.5	48.0	31.0	46.6
ShareGPT4Video	7B	49.5	39.5	79.5	40.0	54.5	82.5	54.5	32.5	50.5	41.5	84.5	35.5	62.5	75.0	51.0	25.5	46.5	28.5	39.0	51.5	51.2
ST-LLM	7B	66.0	53.5	84.0	44.0	58.5	80.5	73.5	38.5	42.5	31.0	86.5	36.5	56.5	78.5	43.0	44.5	46.5	34.5	41.5	58.5	54.9
VideoGPT+	3.8B	69.0	60.0	83.0	48.5	66.5	85.5	75.5	36.0	44.0	34.0	89.5	39.5	71.0	90.5	45.0	53.0	50.0	29.5	44.0	60.0	58.7
VideoChat2	7B	75.5	58.0	83.5	50.5	60.5	87.5	74.5	45.0	47.5	44.0	82.5	37.0	64.5	87.5	51.0	66.5	47.0	35.0	37.0	72.5	60.4
VideoMind (Ours)	2B	77.0	78.0	77.0	46.5	70.5	87.0	71.5	33.0	48.0	39.5	91.0	53.0	78.0	89.0	43.5	53.5	61.5	37.5	49.5	53.0	61.9
VideoMind (Ours)	7B	74.0	71.5	81.0	50.0	77.0	93.0	75.0	38.0	48.5	46.0	91.0	39.0	80.0	94.5	49.5	55.5	70.0	40.5	57.0	61.0	64.6

Method	Size	Acc	(8, 15]	(15, 60]	(180, 600]	(900, 3600]
GPT-4o	–	66.7	71.4	76.7	69.1	60.9
GPT-4 Turbo	–	59.0	65.2	68.2	62.4	50.5
Gemini-1.5-Pro	–	64.0	67.4	75.1	65.3	58.6
Gemini-1.5-Flash	–	61.6	68.3	76.2	62.6	54.0
Idefics2	8B	49.7	59.8	65.7	47.8	42.7
Phi-3-Vision	4B	49.6	59.3	61.6	46.8	44.7
Mantis-Idefics2	8B	47.0	56.6	55.8	45.6	42.2
Mantis-BakLLaVA	7B	43.7	53.4	57.6	40.3	38.7
VideoMind (Ours)	2B	48.8	59.3	59.3	49.3	41.7
VideoMind (Ours)	7B	56.3	67.4	67.4	56.8	48.6