Need help with DCL?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

202 Stars 26 Forks MIT License 14 Commits 2 Opened issues


NeurIPS 2020, Debiased Contrastive Learning

Services available


Need anything else?

Contributors list

# 113,014
11 commits
# 452,305
1 commit

Debiased Contrastive Learning

A prominent technique for self-supervised representation learning has been to contrast semantically similar and dissimilar pairs of samples. Without access to labels, dissimilar (negative) points are typically taken to be randomly sampled datapoints, implicitly accepting that these points may, in reality, actually have the same label. Perhaps unsurprisingly, we observe that sampling negative examples from truly different labels improves performance, in a synthetic setting where labels are available. Motivated by this observation, we develop a debiased contrastive objective that corrects for the sampling of same-label datapoints, even without knowledge of the true labels.

Debiased Contrastive Learning NeurIPS 2020 [paper]
Ching-Yao Chuang, Joshua Robinson, Lin Yen-Chen, Antonio Torralba, and Stefanie Jegelka


  • Python 3.7
  • PyTorch 1.3.1
  • PIL
  • OpenCV

Contrastive Representation Learning

We can train standard (biased) or debiased version (M=1) of SimCLR with
on STL10 dataset.

flags: -

: use debiased objective (True) or standard objective (False) -
: specify class probability -
: batch size for SimCLR

For instance, run the following command to train a debiased encoder.

python --tau_plus = 0.1

*Due to the implementation of
, training with at most 2 GPUs gives the best result.

Linear evaluation

The model is evaluated by training a linear classifier after fixing the learned embedding.

path flags: -

: specify the path to saved model
python --model_path results/model_400.pth

Pretrained Models

| | tauplus | Arch | Latent Dim | Batch Size | Accuracy(%) | Download | |----------|:---:|:----:|:---:|:---:|:---:|:---:| | Biased | tauplus = 0.0 | ResNet50 | 128 | 256 | 80.15 | model| | Debiased |tauplus = 0.05 | ResNet50 | 128 | 256 | 81.85 | model| | Debiased |tauplus = 0.1 | ResNet50 | 128 | 256 | 84.26 | model|


If you find this repo useful for your research, please consider citing the paper

  title={Debiased contrastive learning},
  author={Chuang, Ching-Yao and Robinson, Joshua and Lin, Yen-Chen and Torralba, Antonio and Jegelka, Stefanie},
  journal={Advances in Neural Information Processing Systems},

For any questions, please contact Ching-Yao Chuang ([email protected]).


Part of this code is inspired by leftthomas/SimCLR.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.