by cvqluu

Angular penalty loss functions in Pytorch (ArcFace, SphereFace, Additive Margin, CosFace)

176 Stars 33 Forks Last release: Not found 22 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Angular Penalty Softmax Losses Pytorch

Concise Pytorch implementation of the Angular Penalty Softmax Losses presented in:

  • ArcFace: [1]
  • SphereFace: [2]
  • CosFace/Additive Margin: [3] / [4]

(Note: the SphereFace implementation is not exactly as described in their paper but instead uses the 'trick' presented in the ArcFace paper to use arccosine instead of the double angle formula)

from loss_functions import AngularPenaltySMLoss

in_features = 512 out_features = 10 # Number of classes

criterion = AngularPenaltySMLoss(in_features, out_features, loss_type='arcface') # loss_type in ['arcface', 'sphereface', 'cosface']

Forward method works similarly to nn.CrossEntropyLoss

x of shape (batch_size, in_features), labels of shape (batch_size,)

labels should indicate class of each sample, and should be an int, l satisying 0 <= l < out_dim

loss = criterion(x, labels) loss.backward()


There are a simple set of experiments on Fashion-MNIST [2] included in
which compares the use of ordinary Softmax and Additive Margin Softmax loss functions by projecting embedding features onto a 3D sphere.

The experiments can be run like so:

python --num-epochs 40 --seed 1234 --use-cuda

Which produces the following results:

Baseline (softmax)


Additive Margin Softmax/CosFace




TODO: fix sphereface results

[1] Deng, J. et al. (2018) ‘ArcFace: Additive Angular Margin Loss for Deep Face Recognition’. Available at:

[2] Liu, W. et al. (2017) ‘SphereFace: Deep hypersphere embedding for face recognition’, in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 6738–6746. doi: 10.1109/CVPR.2017.713.

[3] Wang, H. et al. (2018) ‘CosFace: Large Margin Cosine Loss for Deep Face Recognition’. Available at: (Accessed: 12 August 2019).

[4] “Additive Margin Softmax for Face Verification.” Wang, Feng, Jian Cheng, Weiyang Liu and Haijun Liu. IEEE Signal Processing Letters 25 (2018): 926-930.

[5] "Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms." Han Xiao, Kashif Rasul, Roland Vollgraf. arXiv:1708.07747

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.