For the latest update, please go to DeepCriticalLearning

August 7, 2022 · View on GitHub

Deep Robustness: ProSelfLC-CVPR 2021 + Example Weighting (DM+IMAE)

ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks

Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude’s Variance Matters

Derivative Manipulation: Example Weighting via Emphasis Density Funtion in the context of DL

  • Novelty: moving from loss design to derivative design

For any specific discussion or potential future collaboration, please feel free to contact me.

See Citation Details

Please kindly cite the following papers if you find this repo useful.

@inproceddings{wang2021proselflc,
  title={ {ProSelfLC}: Progressive Self Label Correction
  for Training Robust Deep Neural Networks},
  author={Wang, Xinshao and Hua, Yang and Kodirov, Elyor and Clifton, David A and Robertson, Neil M},
  booktitle={CVPR},
  year={2021}
}
@phdthesis{wang2020example,
  title={Example weighting for deep representation learning},
  author={Wang, Xinshao},
  year={2020},
  school={Queen's University Belfast}
}
@article{wang2019derivative,
  title={Derivative Manipulation for General Example Weighting},
  author={Wang, Xinshao and Kodirov, Elyor and Hua, Yang and Robertson, Neil},
  journal={arXiv preprint arXiv:1905.11233},
  year={2019}
}
@article{wang2019imae,
  title={{IMAE} for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude’s Variance Matters},
  author={Wang, Xinshao and Hua, Yang and Kodirov, Elyor and Robertson, Neil M},
  journal={arXiv preprint arXiv:1903.12141},
  year={2019}
}

PyTorch Implementation for ProSelfLC, Derivative Manipulation, Improved MAE

  • Easy to install
  • Easy to use
  • Easy to extend: new losses, new networks, new dataset and loaders
  • Easy to run experiments and sink results
  • Easy to put sinked results into your technical reports and academic papers.

Install

See Install Guidelines

Set the Pipenv From Scratch

  • sudo apt update && sudo apt upgrade
  • sudo apt install python3.8
  • curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
  • python3.8 get-pip.py
  • vim ~/.bashrc -> add export PATH="/home/ubuntu/.local/bin:$PATH" -> source ~/.bashrc
  • pip3 install pipenv

Build env for this repo using pipenv

  • git clone git@github.com:XinshaoAmosWang/ProSelfLC-CVPR2021.git
  • cd ProSelfLC-CVPR2021
  • pipenv install -e .

How to use

Run experiments

  • cd ProSelfLC-CVPR2021
  • pipenv shell
  • python tests/convnets/test_trainer_cnn_vision_cifar100_cce_convets.py
  • python tests/convnets/test_trainer_cnn_vision_cifar100_lscplc_covnets.py
  • python tests/convnets/test_trainer_cnn_vision_cifar100_proselflc_covnets.py
  • python tests/derivativemanipulation_imae/test_dm_imae.py

Visualize results

How to extend this repo

Examples of sinked experimental configs and resutls

See Sinked Results
epochclean_testnoisy_subsetclean_subsetcleaned_noisy_subset
40.18010.00910.1690.1617
80.33290.00890.326550.3065
120.38480.00930.3800250.3451
160.3910.00910.3918750.3561
200.42250.0090.41190.3693
240.42140.00770.4169750.3728
280.42220.01080.430950.3884
320.47090.00970.470.4254
360.41550.00970.429550.3886
400.4490.00830.4631250.4165
440.4480.00740.4411250.4006
480.38560.00780.3830250.3496
520.46720.00830.4794750.4373
560.44280.00810.4370750.3891
600.41640.00950.4226750.3815
640.46350.00790.4832250.4386
680.45060.00850.46540.4145
720.44280.00810.4598250.4105
760.45530.00860.45790.4151
800.61080.01040.6707750.5751
840.59890.01070.673950.5636
880.60260.00980.68320.5703
920.59490.0130.6799250.5608
960.59760.01220.679850.5487
1000.58380.01230.670.5426
1040.5920.01130.674650.5544
1080.6030.01170.67230.5552
1120.580.01170.6621750.5394
1160.57670.01260.6608250.5421
1200.58290.01210.6549250.5382
1240.58280.01270.6558750.5426
1280.58250.0130.6525750.5405
1320.56410.01110.6252750.5303
1360.57790.01120.6352750.5355
1400.64620.01390.761750.6095
1440.64640.01590.769650.6175
1480.64120.01690.7734750.616
1520.64580.01610.7750250.6127
1560.63530.0190.7688250.6036
1600.63850.01680.7681250.6115
1640.63380.01810.7638250.6122
1680.63160.01640.757550.6011
1720.62250.01710.7476750.5942
1760.63120.01530.7494250.5989
1800.65520.0210.79660.6264
1840.6530.0220.80360.6292
1880.65440.02120.8071250.6256
1920.65450.02090.809050.6286
1960.65310.02220.8110750.6276
2000.65720.02290.81020.627
  • params.csv
data_namenum_classesdevicenum_workersbatch_sizecounterlrtotal_epochsmilestonesgammaloss_namesymmetric_noise_ratenetwork_namewarmup_epochsexp_basetransit_time_ratiosummary_writer_dirtraintotal_iterationsmomentumweight_decay
cifar100100gpu4128iteration0.1200600.2proselflc0.2shufflenetv21660.3/home/xinshao/tpami_proselflc_experiments/cifar100_symmetric_noise_rate_0.2/shufflenetv2/050_proselflc_warm16_b6_transit0.3_20210904-172732True782000.90.0005
cifar100100gpu4128iteration0.12001200.2proselflc0.2shufflenetv21660.3/home/xinshao/tpami_proselflc_experiments/cifar100_symmetric_noise_rate_0.2/shufflenetv2/050_proselflc_warm16_b6_transit0.3_20210904-172732True782000.90.0005
cifar100100gpu4128iteration0.12001600.2proselflc0.2shufflenetv21660.3/home/xinshao/tpami_proselflc_experiments/cifar100_symmetric_noise_rate_0.2/shufflenetv2/050_proselflc_warm16_b6_transit0.3_20210904-172732True782000.90.0005

List of Content

  1. Storyline
  2. Open ML Research Questions
  3. Noticeable Findings
  4. Literature Review
  5. In Self LC, a core question is not well answered
  6. Underlying Principle of ProSelfLC
  7. Mathematical Details of ProSelfLC
  8. Design Reasons of ProSelfLC
  9. Related Interesting Work