Need help with awesome-normalization-techniques?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

tczhangzhi
195 Stars 22 Forks 13 Commits 0 Opened issues

Description

Papers for normalization techniques, released codes collections.

Services available

!
?

Need anything else?

Contributors list

# 54,802
model-c...
nas
pytorch
Tensorf...
13 commits

Awesome-Normalization-Techniques Awesome

Papers for normalization techniques, released codes collections.

Any addition or bug feel free to open an issue or pull requests.

2020 - 2019 - 2018 - 2017 - 2016

2020

  • A New Look at Ghost Normalization
    • Neofytos Dimitriou, Ognjen Arandjelovic
    • [Paper]
  • TaskNorm: Rethinking Batch Normalization for Meta-Learning (ICML 2020)
    • John Bronskill, Jonathan Gordon, James Requeima, Sebastian Nowozin, Richard E Turner
    • [Paper]
    • [Python Reference]
  • Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise (AAAI 2020)
  • Towards Stabilizing Batch Statistics in BackWard Propagation of Batch Normalization (ICLR 2020)
    • Junjie Yan, Ruosi Wan, Xiangyu Zhang, Wei Zhang, Yichen Wei, Jian Sun
    • [Paper]
  • Rethinking Spatially-Adaptive Normalization
    • Zhentao Tan, Dongdong Chen, Qi Chu, Menglei Chai, Jing Liao, Mingming He, Lu Yuan, Nenghai Yu
    • [Paper]
  • Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks (CVPR 2020)
  • Extended Batch Normalization
  • Knowledge Distillation via Adaptive Instance Normalization
    • Jing Yang, Brais Martinez, Adrian Bulat, and Georgios Tzimiropoulos
    • [Paper]
  • Four Things Everyone Should Know to Improve Batch Normalization
    • Cecilia Summers, Michael J. Dinneen
    • [Paper]
  • Region Normalization for Image Inpainting (AAAI 2020)
    • Tao Yu, Zongyu Guo, Xin Jin, Shilin Wu, Zhibo Chen, Weiping Li, Zhizheng Zhang, Sen Liu
    • [Paper]
    • [Python Reference]
  • Local Context Normalization: Revisiting Local Normalization (CVPR 2020)
    • Anthony Ortiz , Caleb Robinson, Mahmudulla Hassan, Dan Morris2, Olac Fuentes1, Christopher Kiekintveld, and Nebojsa Jojic
    • [Paper]
  • Attentive Normalization for Conditional Image Generation (CVPR 2020)
  • Cross-Iteration Batch Normalization
  • SEAN: Image Synthesis with Semantic Region-Adaptive Normalization (CVPR 2020)
  • Evolving Normalization-Activation Layers
    • Hanxiao Liu, Andrew Brock, Karen Simonyan, Quoc V. Le
    • [Paper]

2019

  • Differentiable Dynamic Normalization for Learning Deep Representation (PMLR 2019/ICML 2019)
    • Ping Luo, Peng Zhanglin, Shao Wenqi, Zhang Ruimao, Ren Jiamin, Wu Lingyun
    • [Paper]
  • Mean Spectral Normalization of Deep Neural Networks for Embedded Automation
  • An Empirical Study of Batch Normalization and Group Normalization in Conditional Computation
    • Vincent Michalski, Vikram Voleti, Samira Ebrahimi Kahou, Anthony Ortiz, Pascal Vincent, Chris Pal, Doina Precup
    • [Paper]
  • Generalized Batch Normalization: Towards Accelerating Deep Neural Networks (AAAI 2019)
    • Xiaoyong Yuan, Zheng Feng, Matthew Norton, Xiaolin Li
    • [Paper]
  • Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift
    • Michał Zając, Konrad Zolna, Stanisław Jastrzębski
    • [Paper]
  • Channel Normalization in Convolutional Neural Network avoids Vanishing Gradients (ICML 2019)
  • Unpaired Image Translation via Adaptive Convolution-based Normalization
    • Wonwoong Cho, Kangyeol Kim, Eungyeup Kim, Hyunwoo J. Kim, Jaegul Choo
    • [Paper]
  • EvalNorm: Estimating Batch Normalization Statistics for Evaluation (ICCV 2019)
    • Saurabh Singh, Abhinav Shrivastava
    • [Paper]
  • Online Normalization for Training Neural Networks (NIPS 2019)
  • Transferable Normalization: Towards Improving Transferability of Deep Neural Networks (NIPS 2019)
  • Iterative Normalization: Beyond Standardization Towards Efficient Whitening (CVPR 2019)
  • Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (CVPR 2019)
  • Attentive Normalization
  • Rethinking Normalization and Elimination Singularity in Neural Networks
    • Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, Alan Yuille
    • [Paper]
  • Dynamic Instance Normalization for Arbitrary
    • Yongcheng Jing, Xiao Liu, Yukang Ding, Xinchao Wang, Errui Ding, Mingli Song, Shilei Wen
    • [Paper]
  • Semantic Image Synthesis with Spatially-Adaptive Normalization (CVPR 2019)
  • Differentiable Learning-to-Normalize via Switchable Normalization (ICLR 2019)
  • SSN: Learning Sparse Switchable Normalization via Sparsestmax (CVPR 2019)
    • Wenqi Shao, Tianjian Meng, Jingyu Li, Ruimao Zhang, Yudian Li, Xiaogang Wang, Ping Luo
    • [Paper]
  • A Mean Field Theory of Batch Normalization
    • Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, Samuel S. Schoenholz
    • [Paper]
  • Restructuring Batch Normalization to Accelerate CNN Training
    • Wonkyung Jung, Daejin Jung, Byeongho Kim, Sunjung Lee, Wonjong Rhee, Jung Ho Ahn
    • [Paper]
  • A Novel Convolutional Neural Network for Image Steganalysis with Shared Normalization (TMM 2019)
    • Songtao Wu, Sheng-hua Zhong, and Yan Liu
    • [Paper]
  • Training Faster by Separating Modes of Variation in Batch-normalized Models (TPAMI 2019)
    • Mahdi M. Kalayeh, Mubarak Shah
    • [Paper]
  • Uncertainty Estimation via Stochastic Batch Normalization (ISNN 2019)
    • Andrei Atanov, Arsenii Ashukha, Dmitry Molchanov, Kirill Neklyudov, Dmitry Vetrov
    • [Paper]
    • [Python Reference]

2018

  • Decorrelated Batch Normalization (CVPR 2018)
  • MegDet: A Large Mini-Batch Object Detector (CVPR 2018)
    • Chao Peng, Tete Xiao, Zeming Li, Yuning Jiang, Xiangyu Zhang, Kai Jia, Gang Yu, Jian Sun
    • [Paper]
  • Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks (NIPS 2018)
    • Hyeonseob Nam, Hyo-Eun Kim
    • [Paper]
  • Kalman Normalization: Normalizing Internal Representations Across Network Layers (NIPS 2018)
    • Guangrun Wang, Jiefeng Peng, Ping Luo, Xinjiang Wang, Liang Lin
    • [Paper]
  • Do Normalization Layers in a Deep ConvNet Really Need to Be Distinct?
    • Ping Luo, Zhanglin Peng, Jiamin Ren, Ruimao Zhang
    • [Paper]
  • L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks (TNNLS)
    • Shuang Wu , Guoqi Li, Lei Deng, Liu Liu, Dong Wu, Yuan Xie, Luping Shi
    • [Paper]
  • In-Place Activated BatchNorm for Memory-Optimized Training of DNNs (CVPR 2018)
  • Group Normalization (ECCV 2018)
  • Spectral Normalization for Generative Adversarial

2017

  • Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models (NIPS 2017)
  • Modulating Early Visual Processing by Language (NIPS 2017)
    • Harm de Vries, Florian Strub, Jérémie Mary, Hugo Larochelle, Olivier Pietquin, Aaron Courville
    • [Paper]
    • [Python Reference]
  • Instance Normalization: the Missing Ingredient for Fast Stylization
  • Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
    • Luo Chunjie, Zhan Jianfeng, Wang Lei, Yang Qiang
    • [Paper]
  • Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization (ICCV 2017)
  • Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes
    • Mengye Ren, Renjie Liao, Raquel Urtasun, Fabian H. Sinz, Richard S. Zemel􏲗
    • [Paper]
  • Recurrent Batch Normalization
    • Tim Cooijmans, Nicolas Ballas, César Laurent, Çag ̆lar Gülçehre, Aaron Courville
    • [Paper]

2016

  • Layer Normalization

  • Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks

    • Devansh Arpit, Yingbo Zhou, Bhargava U. Kota, Venu Govindaraju, SUNY Buffalo
    • [Paper]
  • Weight Normalization: a Simple Reparameterization to Accelerate Training of Deep Neural Networks (NIPS 2016)

    • Tim Salimans, Diederik P. Kingma
    • [Paper]

2015

  • Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.