model-compression-and-acceleration

July 13, 2018 · View on GitHub

REFERENCE

🌟 Overview

  • [2018-arXiv] Recent Advances in Efficient Computation of Deep Convolutional Neural Networks paper

  • [2018-arXiv] A Survey on Acceleration of Deep Convolutional Neural Networkspaper

  • [2017-arXiv] A Survey of Model Compression and Acceleration for Deep Neural Networks paper

  • [2017-arXiv] Model compression as constrained optimization, with application to neural nets. Part I: general framework paper

  • [2017-arXiv] Model compression as constrained optimization, with application to neural nets. Part II: quantization paper

🌟 Compact Network Design

  • [2018-arXiv] IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks paper

  • [2018-CVPR] IGCV2: Interleaved Structured Sparse Convolutional Neural Networks paper

  • [2018-arXiv] SqueezeNext: Hardware-Aware Neural Network Design paper

  • [2018-CVPR] MobileNetV2: Inverted Residuals and Linear Bottlenecks papercode

  • [2017-CVPR] SENet: Squeeze-and-Excitation Networks paper code

  • [2017-CVPR] MobileNetsV1: Efficient Convolutional Neural Networks for Mobile Vision Applicationspaper code

  • [2017-CVPR] ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicespaper code

  • [2017-CVPR] ResNeXt: Aggregated Residual Transformations for Deep Neural Networks paper code

  • [2017-CVPR] Xception: Deep Learning with Depthwise Separable Convolutions paper code

  • [2017-ICCV] ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression paper code

  • [2017-CVPR] SEP-Nets: Small and Effective Pattern Networks paper

  • [2016-ICLR] SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5MB model size paper code

🌟 Distillation

  • [2018-ICLR] Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification paper

  • [2018-ICLR] Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracypaper

  • [2018-ICLR] Model compression via distillation and quantization paper

  • [2018-AAAI] DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transferpaper

  • [2017-CVPR] Mimicking Very Efficient Network for Object Detection paper

  • [2017-CVPR] Like What You Like: Knowledge Distill via Neuron Selectivity Transferpaper

  • [2017-CVPR] A Gift from Knowledge Distillation:Fast Optimization, Network Minimization and Transfer Learningpaper

  • [2016-AAAI] Face Model Compression by Distilling Knowledge from Neurons paper

  • [2016-CVPR] Cross Model Distillation for Supervision Transfer paper code

  • [2016-ICLR] Net2net: Accelerating learning via knowledge transferpaper

  • [2015-ICLR] FitNets: Hints for Thin Deep Nets paper

🌟 Pruning

  • [2018-ICLR] To prune, or not to prune: exploring the efficacy of pruning for model compression paper 

  • [2018-CVPR] NISP: Pruning Networks using Neuron Importance Score Propagation paper 

  • [2018-CVPR] “Learning-Compression” Algorithms for Neural Net Pruning paper

  • [2018-ICLR] Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers paper 

  • [2018-ICLR] Efficient Sparse-Winograd Convolutional Neural Networks paper 

  • [2018-WACV] Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networks paper 

  • [2018-ICML] Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions paper code

  • [2017-ICCV] Channel Pruning for Accelerating Very Deep Neural Networks paper code

🌟 Binarization

  • [2016-ArXiv] Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 paper code
  • [2016-ECCV] XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks paper code

🌟 Quantization

  • [2018-ICLR] Variational Network Quantizationpaper
  • [2018-AAAI] Deep Neural Network Compression with Single and Multiple Level Quantization paper
  • [2018-CVPR] Google: Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference paper
  • [2018-ArXiv] Google: Quantizing deep convolutional networks for efficient inference: A whitepaperpaper
  • [2018-ICLR] Training and Inference with Integers in Deep Neural Networks paper
  • [2018-arXiv] On the Universal Approximability of Quantized ReLU Neural Networks paper

🌟 Low Rank Approximation

  • [2017-ICCV] Coordinating Filters for Faster Deep Neural Networks paper
  • [2017-NIPS] High performance ultra-low-precision convolutions on mobile devicespaper 

🌟 Accelerating / Fast Algorithms

  • [2018-AAAI] Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networkspaper
  • [2018-arXiv] Uber SBNet: Sparse Blocks Network for Fast Inferencepaper