PySegMetric_EvalToolkit

基于python的图像分割测评工具箱（PSM）

已集成的评估指标

• Pixel Accuracy (PA) is calculated based on the binarized prediction mask and ground-truth:

• F-measure is a metric that comprehensively considers both precision and recall:

• weighted F-measure is proposed to improve the metric F-measure. It assigns different weights (ω) to precision and recall across different errors at different locations, considering the neighborhood information:

• S-measure evaluates the spatial structure similarity by combining the region-aware structural similarity Sr and the object-aware structural similarity So:

• E-measure can jointly capture image level statistics and local pixel matching information:

• IOU is the most common metric for evaluating classification accuracy:

• Dice is a statistic used to gauge the similarity of two samples and become the most used metric in validating medical image segmentation:

• Balanced error rate (BER) is a common metric to evaluate shadow detection performance, where shadow and non-shadow regions contribute equally to the overall performance without considering their relative areas:

• MAE measures the average absolute difference between the prediction and the ground truth pixel by pixel:

分割任务中使用各类评估指标的代表性顶会论文工作

显著性目标检测（Salient Object Detection）

• Self-Supervised Pretraining for RGB-D Salient Object Detection, AAAI 2022.[Fmax, Fw, Sm, Em, MAE]
• Suppress and Balance: A Simple Gated Network for Salient Object Detection, ECCV 2020.[Fmax, Sm, MAE]
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection, ECCV 2020.[Fmax, Fmean, Fw, Sm, Em, MAE]
• Multi-scale Interactive Network for Salient Object Detection, CVPR 2020.[Fmax, Fmean, Fw, Sm, Em, MAE]
• Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection, ECCV 2020.[Fmax, Fmean, Fw, Sm, Em, MAE]

伪装目标检测 （Camouflaged Object Detection）

• Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection, CVPR 2022.[Fmax, Fw, Sm, Em, MAE]
• Uncertainty-Guided Transformer Reasoning for Camouflaged Object Detection, ICCV 2021.[Fw, Sm, Em, MAE]
• Context-aware Cross-level Fusion Network for Camouflaged Object Detection, IJCAI 2021.[Fw, Sm, Em, MAE]
• Uncertainty-aware Joint Salient Object and Camouflaged Object Detection, CVPR 2021.[Fmean, Sm, Em, MAE]
• Camouflaged Object Segmentation with Distraction Mining, CVPR 2021.[Fw, Sm, Em, MAE]

散焦模糊检测 （Defocus Blur Detection）

• Self-generated Defocus Blur Detection via Dual Adversarial Discriminators, CVPR 2021.[Fmean, MAE]
• Self-generated Defocus Blur Detection via Dual Adversarial Discriminators, ECCV 2020.[Fmean, MAE]
• R2MRF: Defocus Blur Detection viaRecurrently Refining Multi-Scale Residual Features, AAAI 2020.[Fmean, MAE]
• Enhancing Diversity of Defocus Blur Detectors via Cross-Ensemble Network, CVPR 2019.[Fmean, MAE]
• Defocus Blur Detection via Multi-Stream Bottom-Top-Bottom Fully Convolutional Network, CVPR 2018.[Fmean, MAE]

阴影检测 （Shadow Detection）

• Robust Shadow Detection by Exploring Effective Shadow Contexts, ACM MM 2021.[Ber]
• Mitigating Intensity Bias in Shadow Detection via Feature Decomposition and Reweighting, ICCV 2021.[Ber]
• Distraction-aware Shadow Detection, CVPR 2019.[Ber]
• ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal, ICCV 2019.[Ber]
• Bidirectional Feature Pyramid Network with Recurrent Attention Residual Modules for Shadow Detection, ECCV 2018.[Ber]

透明目标检测 （Transparent Object Dectection）

• Transfusion: A Novel SLAM Method Focused on Transparent Objects, ICCV 2021.[mIoU, Acc, Ber, MAE]
• Segmenting Transparent Object in the Wild with Transformer, IJCAI 2021.[mIoU, Acc]
• Segmenting Transparent Objects in the Wild, ECCV 2020.[mIoU, Acc, Ber]

玻璃目标检测 （Glass Object Detection）

• Enhanced Boundary Learning for Glass-like Object Segmentation, ICCV 2021.[mIoU, Acc, Fmean, Ber, MAE]
• Rich Context Aggregation with Reflection Prior for Glass Surface Detection, CVPR 2021.[mIoU, Fmean, Ber, MAE]
• Don’t Hit Me! Glass Detection in Real-world Scenes, CVPR 2020.[mIoU, Acc, Fmean, Ber, MAE]

镜子检测 （Mirror Detection）

• Depth-Aware Mirror Segmentation, CVPR 2021.[mIoU, Fw, Ber, MAE]
• Progressive Mirror Detection, CVPR 2020.[Fmean, MAE]
• Where Is My Mirror?, ICCV 2019.[mIoU, Acc, Fmean, Ber, MAE]

息肉分割 （Polyp Segmentation）

• Automatic Polyp Segmentation via Multi-scale Subtraction Network, MICCAI 2021.[mIoU, mDice, Fw, Sm, Em, MAE]
• TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation, MICCAI 2021.[mIoU, mDice]
• Shallow Attention Network for Polyp Segmentation, MICCAI 2021.[mIoU, mDice]
• Learnable Oriented-Derivative Network for Polyp Segmentation, MICCAI 2021.[mIoU, mDice]
• PraNet: Parallel Reverse Attention Network for Polyp Segmentation, MICCAI 2020.[mIoU, mDice, Fw, Sm, Em, MAE]

其他类型分割任务 （人像分割、缺陷检测、表面检测、物体内部探伤检测等）

超快捷使用方法

• 安装./requirements.txt中的依赖
• 对utils/config.py中的Models = {'Model1':Model1,'Model2':Model2}及test_datasets = {'dataset1':dataset1,'dataset2':dataset2}完成对应的路径设置。注意字典中的'dataset'及待测方法中的数据集文件夹名字应与真实的数据集名称保持一致。
• 运行./test_score.py 数值预测结果日志将保留在当前目录中。

评测指标参考文献

@inproceedings{Fmax_mean,
  title={Frequency-tuned salient region detection},
  author={Achanta, Radhakrishna and Hemami, Sheila and Estrada, Francisco and S{\"u}sstrunk, Sabine},
  booktitle= CVPR,
  pages={1597--1604},
  year={2009}
}  

@inproceedings{Fwb,
  title={How to evaluate foreground maps?},
  author={Margolin, Ran and Zelnik-Manor, Lihi and Tal, Ayellet},
  booktitle=CVPR,
  pages={248--255},
  year={2014}
}

@inproceedings{Sm,
  title={Structure-measure: A new way to evaluate foreground maps},
  author={Fan, Deng-Ping and Cheng, Ming-Ming and Liu, Yun and Li, Tao and Borji, Ali},
  booktitle= ICCV,
  pages={4548--4557},
  year={2017}
}  
@inproceedings{Em,
    title="Enhanced-alignment Measure for Binary Foreground Map Evaluation",
    author="Deng-Ping {Fan} and Cheng {Gong} and Yang {Cao} and Bo {Ren} and Ming-Ming {Cheng} and Ali {Borji}",
    booktitle=IJCAI,
    pages="698--704",
    year={2018}
}