Large-Margin Softmax Loss, Angular Softmax Loss, Additive Margin Softmax, ArcFaceLoss And FocalLoss In Tensorflow

This repository contains core codes of the reimplementation of the following papers in TensorFlow:

• Large-Margin Softmax Loss for Convolutional Neural Networks
• SphereFace: Deep Hypersphere Embedding for Face Recognition
• Additive Margin Softmax for Face Verification or CosFace: Large Margin Cosine Loss for Deep Face Recognition
• ArcFace: Additive Angular Margin Loss for Deep Face Recognition
• Focal Loss for Dense Object Detection

If your goal is to reproduce the results in the original paper, please use the official codes:

• Large Margin Softmax Loss in ICML 2016
• Angular Softmax Loss in CVPR 2017
• Additive Margin Softmax
• ArcFace: Additive Angular Margin Loss
• Focal Loss in ICCV 2017

For using these Ops on your own machine:

• copy the header file "cuda_config.h" from "your_python_path/site-packages/external/local_config_cuda/cuda/cuda/cuda_config.h" to "your_python_path/site-packages/tensorflow/include/tensorflow/stream_executor/cuda/cuda_config.h".

• run the following script:

mkdir build
cd build && cmake ..
make

• run "test_op.py" and check the numeric errors to test your install

• follow the below codes snippet to integrate this Op into your own code:

  • For Large Margin Softmax Loss:

  op_module = tf.load_op_library(so_lib_path)
  large_margin_softmax = op_module.large_margin_softmax
  
  @ops.RegisterGradient("LargeMarginSoftmax")
  def _large_margin_softmax_grad(op, grad, _):
    '''The gradients for `LargeMarginSoftmax`.
    '''
    inputs_features = op.inputs[0]
    inputs_weights = op.inputs[1]
    inputs_labels = op.inputs[2]
    cur_lambda = op.outputs[1]
    margin_order = op.get_attr('margin_order')
  
    grads = op_module.large_margin_softmax_grad(inputs_features, inputs_weights, inputs_labels, grad, cur_lambda[0], margin_order)
    return [grads[0], grads[1], None, None]
  
  var_weights = tf.Variable(initial_value, trainable=True, name='lsoftmax_weights')
  result = large_margin_softmax(features, var_weights, labels, global_step, 4, 1000., 0.000025, 35., 0.)
  loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=result[0]))
  

  • For Angular Softmax Loss:

  op_module = tf.load_op_library(so_lib_path)
  angular_softmax = op_module.angular_softmax
  
  @ops.RegisterGradient("AngularSoftmax")
  def _angular_softmax_grad(op, grad, _):
    '''The gradients for `AngularSoftmax`.
    '''
    inputs_features = op.inputs[0]
    inputs_weights = op.inputs[1]
    inputs_labels = op.inputs[2]
    cur_lambda = op.outputs[1]
    margin_order = op.get_attr('margin_order')
  
    grads = op_module.angular_softmax_grad(inputs_features, inputs_weights, inputs_labels, grad, cur_lambda[0], margin_order)
    return [grads[0], grads[1], None, None]
  
  var_weights = tf.Variable(initial_value, trainable=True, name='asoftmax_weights')
  normed_var_weights = tf.nn.l2_normalize(var_weights, 1, 1e-10, name='weights_normed')
  result = angular_softmax(features, normed_var_weights, labels, global_step, 4, 1000., 0.000025, 35., 0.)
  loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=result[0]))
  

  • For others just refer to this script.

All the codes was tested under TensorFlow 1.6, Python 3.5, Ubuntu 16.04 with CUDA 8.0. The outputs of these Ops in C++ had been compared with the original caffe codes' outputs, and the bias could be ignored. The gradients of this Op had been checked using tf.test.compute_gradient_error and tf.test.compute_gradient. While the others are implemented following the official implementation in Python Ops.

If you encountered some linkage problem when generating or loading *.so, you are highly recommended to read this section in the official tourial to make sure you were using the same C++ ABI version.

Any contributions to this repo is welcomed.

MIT License