ReActNet

This is the pytorch implementation of our paper "ReActNet: Towards Precise Binary NeuralNetwork with Generalized Activation Functions", published in ECCV 2020.

In this paper, we propose to generalize the traditional Sign and PReLU functions to RSign and RPReLU, which enable explicit learning of the distribution reshape and shift at near-zero extra cost. By adding simple learnable bias, ReActNet achieves 69.4% top-1 accuracy on Imagenet dataset with both weights and activations being binary, a near ResNet-level accuracy.

Citation

If you find our code useful for your research, please consider citing:

@inproceedings{liu2020reactnet,
  title={ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions},
  author={Liu, Zechun and Shen, Zhiqiang and Savvides, Marios and Cheng, Kwang-Ting},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2020}
}

Run

1. Requirements:

• python3, pytorch 1.4.0, torchvision 0.5.0

2. Data:

• Download ImageNet dataset

3. Steps to run:

(1) Step1: binarizing activations

• Change directory to ./resnet/1_step1/ or ./mobilenet/1_step1/
• run bash run.sh

(2) Step2: binarizing weights + activations

• Change directory to ./resnet/2_step2/ or ./mobilenet/2_step2/
• run bash run.sh

Models

Methods	Top1-Acc	FLOPs	Trained Model
XNOR-Net	51.2%	1.67 x $10^{8}$	-
Bi-Real Net	56.4%	1.63 x $10^{8}$	-
Real-to-Binary	65.4%	1.83 x $10^{8}$	-
ReActNet (Bi-Real based)	65.9%	1.63 x $10^{8}$	Model-ReAct-ResNet
ReActNet-A	69.5%	0.87 x $10^{8}$	Model-ReAct-MobileNet

Contact

Zechun Liu, HKUST (zliubq at connect.ust.hk)

Zhiqiang Shen, CMU (zhiqians at andrew.cmu.edu)