supported_metrics.md

August 5, 2026 · View on GitHub

Independent Metrics

We include x mark if the metric is auto-installed in versa.

Number	Auto-Install	Metric Name (Auto-Install)	Key in config	Key in report	Code Source	References
1	x	Deep Noise Suppression MOS Score of P.835 (DNSMOS)	pseudo_mos	dnsmos_overall	speechmos (MS)	paper
2	x	Deep Noise Suppression MOS Score of P.808 (DNSMOS)	pseudo_mos	dnsmos_p808	speechmos (MS)	paper
3	x	Non-intrusive Speech Quality and Naturalness Assessment (NISQA)	nisqa	{nisqa_mos_pred, nisqa_noi_pred, nisqa_dis_pred, nisqa_col_pred, nisqa_loud_pred}	NISQA	paper
4	x	UTokyo-SaruLab System for VoiceMOS Challenge 2022 (UTMOS)	pseudo_mos	utmos	speechmos	paper
5	x	Packet Loss Concealment-related MOS Score (PLCMOS)	pseudo_mos	plcmos	speechmos (MS)	paper
6	x	PESQ in TorchAudio-Squim	squim_no_ref	torch_squim_pesq	torch_squim	paper
7	x	STOI in TorchAudio-Squim	squim_no_ref	torch_squim_stoi	torch_squim	paper
8	x	SI-SDR in TorchAudio-Squim	squim_no_ref	torch_squim_si_sdr	torch_squim	paper
9	x	Singing voice MOS	singmos	singmos	singmos	paper
10	x	Sheet SSQA MOS Models	sheet_ssqa	sheet_ssqa	Sheet	paper
11		UTMOSv2: UTokyo-SaruLab MOS Prediction System	utmosv2	utmosv2	UTMOSv2	paper
12		Speech Contrastive Regression for Quality Assessment without reference (ScoreQ)	scoreq_nr	scoreq_nr	ScoreQ	paper
13	x	Speech enhancement-based SI-SNR	se_snr	se_si_snr	ESPnet
14	x	Speech enhancement-based CI-SDR	se_snr	se_ci_sdr	ESPnet
15	x	Speech enhancement-based SAR	se_snr	se_sar	ESPnet
16	x	Speech enhancement-based SDR	se_snr	se_sdr	ESPnet
17	x	PAM: Prompting Audio-Language Models for Audio Quality Assessment	pam	pam	PAM	Paper
18		Speech-to-Reverberation Modulation energy Ratio (SRMR)	srmr	srmr	SRMRpy	Paper
19	x	Voice Activity Detection (VAD)	vad	vad_info	SileroVAD
20		Speaker Turn Taking (SPK-TT)
21	x	Speaking Word/Character Rate (SWR)	speaking_rate	speaking_rate	-	-
22	x	Auti-spoofing Score (SpoofS) with AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks	asvspoof_score	asvspoof_score	AASIST	Paper
23	x	Language Identification	lid	language	ESPnet	Paper
24		Audiobox Aesthetics	audiobox_aesthetics	{audiobox_aesthetics_CE, audiobox_aesthetics_CU, audiobox_aesthetics_PC, audiobox_aesthetics_PQ}	Audiobox-Aesthetics	Paper
25	x	Qwen2 Speaker Characteristics - Count	qwen2_speaker_count_metric	qwen2_speaker_count_metric	Qwen2 Audio	paper
26	x	Qwen2 Speaker Characteristics - Gender	qwen2_speaker_gender_metric	qwen2_speaker_gender_metric	Qwen2 Audio	paper
27	x	Qwen2 Speaker Characteristics - Age	qwen2_speaker_age_metric	qwen2_speaker_age_metric	Qwen2 Audio	paper
28	x	Qwen2 Speaker Characteristics - Speech Impairment	qwen2_speech_impairment_metric	qwen2_speech_impairment_metric	Qwen2 Audio	paper
29	x	Qwen2 Voice Properties - Pitch	qwen2_voice_pitch_metric	qwen2_voice_pitch_metric	Qwen2 Audio	paper
30	x	Qwen2 Voice Properties - Pitch Range	qwen2_pitch_range_metric	qwen2_pitch_range_metric	Qwen2 Audio	paper
31	x	Qwen2 Voice Properties - Voice Type	qwen2_voice_type_metric	qwen2_voice_type_metric	Qwen2 Audio	paper
32	x	Qwen2 Voice Properties - Volume Level	qwen2_speech_volume_level_metric	qwen2_speech_volume_level_metric	Qwen2 Audio	paper
33	x	Qwen2 Speech Content - Language	qwen2_language_metric	qwen2_language_metric	Qwen2 Audio	paper
34	x	Qwen2 Speech Content - Register	qwen2_speech_register_metric	qwen2_speech_register_metric	Qwen2 Audio	paper
35	x	Qwen2 Speech Content - Vocabulary Complexity	qwen2_vocabulary_complexity_metric	qwen2_vocabulary_complexity_metric	Qwen2 Audio	paper
36	x	Qwen2 Speech Content - Purpose	qwen2_speech_purpose_metric	qwen2_speech_purpose_metric	Qwen2 Audio	paper
37	x	Qwen2 Speech Delivery - Emotion	qwen2_speech_emotion_metric	qwen2_speech_emotion_metric	Qwen2 Audio	paper
38	x	Qwen2 Speech Delivery - Clarity	qwen2_speech_clarity_metric	qwen2_speech_clarity_metric	Qwen2 Audio	paper
39	x	Qwen2 Speech Delivery - Rate	qwen2_speech_rate_metric	qwen2_speech_rate_metric	Qwen2 Audio	paper
40	x	Qwen2 Speech Delivery - Style	qwen2_speaking_style_metric	qwen2_speaking_style_metric	Qwen2 Audio	paper
41	x	Qwen2 Speech Delivery - Emotional Vocalizations	qwen2_laughter_crying_metric	qwen2_laughter_crying_metric	Qwen2 Audio	paper
42	x	Qwen2 Interaction Patterns - Overlapping Speech	qwen2_overlapping_speech_metric	qwen2_overlapping_speech_metric	Qwen2 Audio	paper
43	x	Qwen2 Recording Environment - Background	qwen2_speech_background_environment_metric	qwen2_speech_background_environment_metric	Qwen2 Audio	paper
44	x	Qwen2 Recording Environment - Quality	qwen2_recording_quality_metric	qwen2_recording_quality_metric	Qwen2 Audio	paper
45	x	Qwen2 Recording Environment - Channel Type	qwen2_channel_type_metric	qwen2_channel_type_metric	Qwen2 Audio	paper
46	x	Qwen2.5-Omni Speech and Singing Characteristics	qwen_omni_{sub_metrics}, qwen_omni_{sub_metrics}_metric	qwen_omni_{sub_metrics}	Qwen2.5-Omni	-
47	x	Dimensional Emotion	emo_vad	arousal_emo_vad, valence_emo_vad, dominance_emo_vad	w2v2-how-to	paper
48	x	Uni-VERSA (Versatile Speech Assessment with a Unified Framework)	universa, universa_noref, universa_audioref, universa_textref, universa_fullref	universa_{sub_metrics}	Uni-VERSA	paper
49		ARECHO (Audio Reference Echo Cancellation and Codec Quality Assessment) - No Reference	arecho, arecho_noref	arecho_{sub_metrics}	ARECHO	paper
50	x	DNSMOS Pro: A Reduced-Size DNN for Probabilistic MOS of Speech	pseudo_mos	dnsmos_pro_bvcc	DNSMOSPro	paper
51	x	DNSMOS Pro: A Reduced-Size DNN for Probabilistic MOS of Speech	pseudo_mos	dnsmos_pro_nisqa	DNSMOSPro	paper
52	x	DNSMOS Pro: A Reduced-Size DNN for Probabilistic MOS of Speech	pseudo_mos	dnsmos_pro_vcc2018	DNSMOSPro	paper
53		WV-MOS (MOS score prediction by fine-tuned wav2vec2.0 model)	wvmos	wvmos	wvmos	paper
54		SIG-MOS	sigmos	{SIGMOS_COL, SIGMOS_DISC, SIGMOS_LOUD, SIGMOS_REVERB, SIGMOS_SIG, SIGMOS_OVRL}	sigmos	paper
55	x	VQScore (Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech)	vqscore	vqscore	VQScore	paper
56	x	Singing voice MOS	pseudo_mos	singmos_pro	singmos	paper
57		SongEval (A Benchmark Dataset for Song Aesthetics Evaluation)	songeval	{songeval_coherence, songeval_musicality, songeval_memorability, songeval_clarity, songeval_naturalness}	SongEval	paper
58		Multivariate Probabilistic Assessment of Speech Quality	multigauss	multigauss_{mos,noi,col,dis,loud}, multigauss_covariance	MultiGauss	paper

Dependent Metrics

Number	Auto-Install	Metric Name (Auto-Install)	Key in config	Key in report	Code Source	References
1	x	Mel Cepstral Distortion (MCD)	mcd_f0	mcd	espnet and s3prl-vc	paper
2	x	F0 Correlation	mcd_f0	f0_corr	espnet and s3prl-vc	paper
3	x	F0 Root Mean Square Error	mcd_f0	f0_rmse	espnet and s3prl-vc	paper
4	x	Signal-to-interference Ratio (SIR)	signal_metric	sir	espnet	-
5	x	Signal-to-artifact Ratio (SAR)	signal_metric	sar	espnet	-
6	x	Signal-to-distortion Ratio (SDR)	signal_metric	sdr	espnet	-
7	x	Convolutional scale-invariant signal-to-distortion ratio (CI-SDR)	signal_metric	ci-sdr	ci_sdr	paper
8	x	Scale-invariant signal-to-noise ratio (SI-SNR)	signal_metric	si-snr	espnet	paper
9	x	Perceptual Evaluation of Speech Quality (PESQ)	pesq	pesq	pesq	paper
10	x	Short-Time Objective Intelligibility (STOI)	stoi	stoi	pystoi	paper
11	x	Speech BERT Score	discrete_speech	speech_bert	discrete speech metric	paper
12	x	Discrete Speech BLEU Score	discrete_speech	speech_belu	discrete speech metric	paper
13	x	Discrete Speech Token Edit Distance	discrete_speech	speech_token_distance	discrete speech metric	paper
14		Dynamic Time Warping Cost Metric	warpq	warpq	WARP-Q	paper
15		Speech Contrastive Regression for Quality Assessment with reference (ScoreQ)	scoreq_ref	scoreq_ref	ScoreQ	paper
16		2f-Model
17	x	Log-Weighted Mean Square Error	log_wmse	log_wmse	log_wmse
18	x	ASR-oriented Mismatch Error Rate (ASR-Mismatch)	asr_match	asr_match_error_rate	-	-
19		Virtual Speech Quality Objective Listener (VISQOL)	visqol	visqol	google-visqol	paper
20		Frequency-Weighted SEGmental SNR (FWSEGSNR)	pysepm	pysepm_fwsegsnr	pysepm	Paper
21		Weighted Spectral Slope (WSS)	pysepm	pysepm_wss	pysepm	Paper
22		Cepstrum Distance Objective Speech Quality Measure (CD)	pysepm	pysepm_cd	pysepm	Paper
23		Composite Objective Speech Quality (composite)	pysepm	pysepm_Csig, pysepm_Cbak, pysepm_Covl	pysepm	Paper
24		Coherence and speech intelligibility index (CSII)	pysepm	pysepm_csii_high, pysepm_csii_mid, pysepm_csii_low	pysepm	Paper
25		Normalized-covariance measure (NCM)	pysepm	pysepm_ncm	pysepm	Paper
26	x	Uni-VERSA (Versatile Speech Assessment with a Unified Framework) with Paired Reference	universa	universa_{sub_metrics}	Uni-VERSA	paper
27	x	Chroma-related Alignment	chroma_alignment	chroma_{stft,cqt,cens}_{cosine, euclidean}_dtw{"", _log, _raw}	-	-
28	x	Deep Perceptual Audio Metric (DPAM)	dpam	dpam_distance	PerceptualAudio_Pytorch	paper
29	x	Contrastive learning-based Deep Perceptual Audio Metric (CDPAM)	cdpam	cdpam_distance	PerceptualAudio	paper

Non-match Metrics

Number	Auto-Install	Metric Name (Auto-Install)	Key in config	Key in report	Code Source	References
1		NORESQA : A Framework for Speech Quality Assessment using Non-Matching References	noresqa	noresqa	Noresqa	Paper
2	x	MOS in TorchAudio-Squim	squim_ref	torch_squim_mos	torch_squim	paper
3	x	ESPnet Speech Recognition-based Error Rate	espnet_wer	espnet_wer	ESPnet	paper
4	x	ESPnet-OWSM Speech Recognition-based Error Rate	owsm_wer	owsm_wer	ESPnet	paper
5	x	OpenAI-Whisper Speech Recognition-based Error Rate (WER/CER, optional PER)	whisper_wer	whisper_wer	Whisper	paper
6		Faster-Whisper Speech Recognition-based Error Rate	fwhisper_wer	fwhisper_wer	Faster-Whisper	-
7		NVIDIA Conformer-Transducer X-Large Speech Recognition-based Error Rate	nemo_wer	nemo_wer	NeMo	paper
8	x	Facebook Hubert-Large-Finetuned Speech Recognition-based Error Rate	hubert_wer	hubert_wer	HuBERT	paper
9		Emotion2vec similarity (emo2vec)	emo2vec_similarity	emotion_similarity	emo2vec	paper
10	x	Speaker Embedding Similarity	speaker	spk_similarity	espnet, transformers x-vector (e.g. WavLM)	paper
11		NOMAD: Unsupervised Learning of Perceptual Embeddings For Speech Enhancement and Non-Matching Reference Audio Quality Assessment	nomad	nomad	Nomad	paper
12		Contrastive Language-Audio Pretraining Score (CLAP Score)	clap_score	clap_score	frechet-audio-distance	paper
13		Accompaniment Prompt Adherence (APA)	apa	apa	Sony-audio-metrics	paper
14		Log Likelihood Ratio (LLR)	pysepm	pysepm_llr	pysepm	Paper
15	x	Uni-VERSA (Versatile Speech Assessment with a Unified Framework) with Paired Text	universa	universa_{sub_metrics}	Uni-VERSA	paper
16		Singer Embedding Similarity	singer	singer_similarity	SSL-Singer-Identity	paper

Distributional Metrics (in verifying)

Number	Metric Name (Auto-Install)	Key in config	Key in report	Code Source	References
1	Frechet Audio Distance (FAD)	fad	fad	fadtk	paper
2	Individual Frechet Audio Distance	individual_fad	individual_fad	fadtk	paper
3	Kullback-Leibler Divergence on Embedding Distribution	kl_embedding	kl_embedding	Stability-AI
4	Audio Density Score	audio_density_coverage	audio_density	Sony-audio-metrics	paper
5	Audio Coverage Score	audio_density_coverage	audio_coverage	Sony-audio-metrics	paper
6	KID : Kernel Distance Metric for Audio/Music Quality	kid	kid_mean, kid_std	KID	Paper

Acknowledgement

We sincerely thank all the open-source implementations listed in https://github.com/shinjiwlab/versa/tree/main#list-of-metrics