Backdoor Learning Resources

A curated list of Backdoor Learning resources. For more details and our categorization criteria, please refer to our survey.

Why Backdoor Learning?

Backdoor learning is an emerging research area, which discusses the security issues of the training process towards machine learning algorithms. It is critical for safely adopting third-party algorithms in reality. Although backdoor learning shares certain similarity with adversarial learning (which concentrates on the security issues of the inference process), they do have essential differences and can be easily distinguished.

Note: 'Backdoor' is also commonly called the 'Neural Trojan' or 'Trojan'.

Contributing

Please help to contribute this list by contacting me or add pull request

Markdown format:

markdown
- Paper Name. 
  [[pdf]](link) 
  [[code]](link)
  - Author 1, Author 2, and Author 3. *Conference/Journal*, Year.

Table of Contents

• Related Survey
• Image and Video Classification
  • Poisoning-based Attack
  • Non-poisoning-based Attack
  • Backdoor Defense
• Attack and Defense Towards Other Tasks and Paradigms
  
• Properties Discussion and Evaluation
• Backdoor Attack for Good
• Toolbox
• Competition

Related Survey

• Backdoor Learning: A Survey. [pdf]

  • Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. arXiv, 2020.

• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review. [pdf]

  • Yansong Gao, Bao Gia Doan, Zhi Zhang, Siqi Ma, Anmin Fu, Surya Nepal, and Hyoungshick Kim. arXiv, 2020.

• Data Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses. [pdf]

  • Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, and Tom Goldstein. arXiv, 2020.

• Deep Learning Backdoors. [pdf]

  • Shaofeng Li, Shiqing Ma, Minhui Xue, and Benjamin Zi Hao Zhao. arXiv, 2020.

• A Survey on Neural Trojans. [pdf]

  • Yuntao Liu, Ankit Mondal, Abhishek Chakraborty, Michael Zuzak, Nina Jacobsen, Daniel Xing, and Ankur Srivastava. ISQED, 2020.

Image and Video Classification

Poisoning-based Attack

2021

• A Master Key Backdoor for Universal Impersonation Attack against DNN-based Face Verification. [link]

  • WeiGuo, Benedetta Tondi, and Mauro Barni. Pattern Recognition Letters, 2021.

• WaNet - Imperceptible Warping-based Backdoor Attack. [pdf]

  • Tuan Anh Nguyen, and Anh Tuan Tran. ICLR, 2021.

• Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxification. [pdf] [code]

  • Siyuan Cheng, Yingqi Liu, Shiqing Ma, and Xiangyu Zhang. AAAI, 2021.

• Backdoors Hidden in Facial Features: A Novel Invisible Backdoor Attack against Face Recognition Systems. [link]

  • Mingfu Xue, Can He, Jian Wang, and Weiqiang Liu. Peer-to-Peer Networking and Applications, 2021.

2020

• One-to-N & N-to-One: Two Advanced Backdoor Attacks against Deep Learning Models. [pdf]

  • Mingfu Xue, Can He, Jian Wang, and Weiqiang Liu. IEEE Transactions on Dependable and Secure Computing, 2020.

• Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization. [pdf] [arXiv Version (2019)]

  • Shaofeng Li, Minhui Xue, Benjamin Zi Hao Zhao, Haojin Zhu, and Xinpeng Zhang. IEEE Transactions on Dependable and Secure Computing, 2020.

• Composite Backdoor Attack for Deep Neural Network by Mixing Existing Benign Features. [pdf]

  • Junyu Lin, Lei Xu, Yingqi Liu, Xiangyu Zhang. CCS, 2020.

• Input-Aware Dynamic Backdoor Attack. [pdf] [code]

  • Anh Nguyen, and Anh Tran. NeurIPS 2020.

• Hidden Trigger Backdoor Attacks. [pdf] [code]

  • Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. AAAI, 2020.

• Bypassing Backdoor Detection Algorithms in Deep Learning. [pdf]

  • Te Juin Lester Tan, and Reza Shokri. EuroS&P, 2020.

• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation. [pdf]

  • Cong Liao, Haoti Zhong, Anna Squicciarini, Sencun Zhu, and David Miller. ACM CODASPY, 2020.

• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks. [pdf] [code]

  • Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. ECCV, 2020.

• Can Adversarial Weight Perturbations Inject Neural Backdoors? [pdf]

  • Siddhant Garg, Adarsh Kumar, Vibhor Goel, and Yingyu Liang. CIKM, 2020.

• Clean-Label Backdoor Attacks on Video Recognition Models. [pdf] [code]

  • Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. CVPR, 2020.

• Escaping Backdoor Attack Detection of Deep Learning. [link]

  • Yayuan Xiong, Fengyuan Xu, Sheng Zhong, and Qun Li. IFIP SEC, 2020.

• Live Trojan Attacks on Deep Neural Networks. [pdf] [code]

  • Robby Costales, Chengzhi Mao, Raphael Norwitz, Bryan Kim, and Junfeng Yang. CVPR Workshop, 2020.

• Backdooring and Poisoning Neural Networks with Image-Scaling Attacks. [pdf]

  • Erwin Quiring, and Konrad Rieck. IEEE S&P Workshop, 2020.

• Backdoor Attack with Sample-Specific Triggers. [pdf]

  • Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. arXiv, 2020.

• Blind Backdoors in Deep Learning Models. [pdf]

  • Eugene Bagdasaryan, and Vitaly Shmatikov. arXiv, 2020.

• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios. [pdf]

  • Hassan Ali, Surya Nepal, Salil S. Kanhere, and Sanjay Jha. arXiv, 2020.

• FaceHack: Triggering Backdoored Facial Recognition Systems Using Facial Characteristics. [pdf]

  • Esha Sarkar, Hadjer Benkraouda, and Michail Maniatakos. arXiv, 2020.

• Light Can Hack Your Face! Black-box Backdoor Attack on Face Recognition Systems. [pdf]

  • Haoliang Li, Yufei Wang, Xiaofei Xie, Yang Liu, Shiqi Wang, Renjie Wan, Lap-Pui Chau, and Alex C. Kot. arXiv, 2020.

• Class-Oriented Poisoning Attack. [pdf]

  • Bingyin Zhao, and Yingjie Lao. arXiv, 2020.

• Dynamic Backdoor Attacks Against Machine Learning Models. [pdf]

  • Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. arXiv, 2020.
    

2019

• Latent Backdoor Attacks on Deep Neural Networks. [pdf]

  • Yuanshun Yao, Huiying Li, Haitao Zheng and Ben Y. Zhao. CCS, 2019.

• A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning. [pdf]

  • M.Barni, K.Kallas, and B.Tondi. ICIP, 2019.

• Label-Consistent Backdoor Attacks. [pdf] [code]

  • Alexander Turner, Dimitris Tsipras, and Aleksander Madry. arXiv, 2019.

2017

• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain. [pdf] [journal]

  • Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. arXiv, 2017 (IEEE Access, 2019).

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. [pdf] [code]

  • Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. arXiv, 2017.

• Trojaning Attack on Neural Networks. [pdf]

  • Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, and Juan Zhai. NDSS, 2017.

Non-poisoning-based Attack

• An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks. [pdf] [code]

  • Ruixiang Tang, Mengnan Du, Ninghao Liu, Fan Yang, and Xia Hu. KDD, 2020.

• TBT: Targeted Neural Network Attack with Bit Trojan. [pdf] [code]

  • Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. CVPR, 2020.

• DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection. [pdf]

  • Yuanchun Li, Jiayi Hua, Haoyu Wang, Chunyang Chen, and Yunxin Liu. ICSE, 2021.

• TrojanNet: Embedding Hidden Trojan Horse Models in Neural Network. [pdf]

  • Chuan Guo, Ruihan Wu, and Kilian Q. Weinberger. arXiv, 2020.

• Don't Trigger Me! A Triggerless Backdoor Attack Against Deep Neural Networks. [pdf]

  • Ahmed Salem, Michael Backes, and Yang Zhang. arXiv, 2020.

• Backdooring Convolutional Neural Networks via Targeted Weight Perturbations. [pdf]

  • Jacob Dumford, and Walter Scheirer. arXiv, 2018.

Backdoor Defense

Preprocessing based Empirical Defense

• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation. [pdf]

  • Yi Zeng, Han Qiu, Shangwei Guo, Tianwei Zhang, Meikang Qiu, and Bhavani Thuraisingham. AsiaCCS, 2021.

• Neural Trojans. [pdf]

  • Yuntao Liu, Yang Xie, and Ankur Srivastava. ICCD, 2017.

• Rethinking the Trigger of Backdoor Attack. [pdf]

  • Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. arXiv, 2020.

• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks. [pdf]

  • Miguel Villarreal-Vasquez, and Bharat Bhargava. arXiv, 2021.

• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems. [pdf]

  • Bao Gia Doan, Ehsan Abbasnejad, and Damith C. Ranasinghe. arXiv, 2019.

• Model Agnostic Defense against Backdoor Attacks in Machine Learning. [pdf]

  • Sakshi Udeshi, Shanshan Peng, Gerald Woo, Lionell Loh, Louth Rawshan, and Sudipta Chattopadhyay. arXiv, 2019.

Model Reconstruction based Empirical Defense

• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks. [pdf] [code]

  • Yige Li, Xingjun Ma, Nodens Koren, Lingjuan Lyu, Xixiang Lyu, and Bo Li. ICLR, 2021.

• Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness. [pdf] [code]

  • Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. ICLR, 2020.

• Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. [pdf] [code]

  • Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. RAID, 2018.
    

• Neural Trojans. [pdf]

  • Yuntao Liu, Yang Xie, and Ankur Srivastava. ICCD, 2017.

• Disabling Backdoor and Identifying Poison Data by using Knowledge Distillation in Backdoor Attacks on Deep Neural Networks. [pdf]

  • Kota Yoshida, and Takeshi Fujino. CCS Workshop, 2020.

• Defending against Backdoor Attack on Deep Neural Networks. [pdf]

  • Hao Cheng, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Pu Zhao, and Xue Lin. KDD Workshop, 2019.

• Neural Network Laundering: Removing Black-Box Backdoor Watermarks from Deep Neural Networks. [pdf]

  • William Aiken, Hyoungshick Kim, and Simon Woo. arXiv, 2020.

• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios. [pdf]

  • Hassan Ali, Surya Nepal, Salil S. Kanhere, and Sanjay Jha. arXiv, 2020.

Trigger Synthesis based Empirical Defense

• Detection of Backdoors in Trained Classifiers Without Access to the Training Set. [pdf]

  • Z Xiang, DJ Miller, and G Kesidis. IEEE Transactions on Neural Networks and Learning Systems, 2020.

• Towards Inspecting and Eliminating Trojan Backdoors in Deep Neural Networks. [pdf] [previous version] [code]

  • Wenbo Guo, Lun Wang, Xinyu Xing, Min Du, and Dawn Song. ICDM, 2020.

• GangSweep: Sweep out Neural Backdoors by GAN. [pdf]

  • Liuwan Zhu, Rui Ning, Cong Wang, Chunsheng Xin, and Hongyi Wu. ACM MM, 2020.

• Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks. [pdf] [code]

  • Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, Ben Y. Zhao. IEEE S&P, 2019。

• Defending Neural Backdoors via Generative Distribution Modeling. [pdf] [code]

  • Ximing Qiao, Yukun Yang, and Hai Li. NeurIPS, 2019.

• DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks. [pdf]

  • Huili Chen, Cheng Fu, Jishen Zhao, Farinaz Koushanfar. IJCAI, 2019.

• Revealing Perceptible Backdoors in DNNs Without the Training Set via the Maximum Achievable Misclassification Fraction Statistic. [pdf]

  • Zhen Xiang, David J. Miller, Hang Wang, and George Kesidis. MLSP, 2020.

• Backdoor Scanning for Deep Neural Networks through K-Arm Optimization. [pdf]

  • Guangyu Shen, Yingqi Liu, Guanhong Tao, Shengwei An, Qiuling Xu, Siyuan Cheng, Shiqing Ma, and Xiangyu Zhang. arXiv, 2021.

• Scalable Backdoor Detection in Neural Networks. [pdf]

  • Haripriya Harikumar, Vuong Le, Santu Rana, Sourangshu Bhattacharya, Sunil Gupta, and Svetha Venkatesh. arXiv, 2020.

• NNoculation: Broad Spectrum and Targeted Treatment of Backdoored DNNs. [pdf] [code]

  • Akshaj Kumar Veldanda, Kang Liu, Benjamin Tan, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg. arXiv, 2020.

Model Diagnosis based Empirical Defense

• Detecting AI Trojans Using Meta Neural Analysis. [pdf]

  • Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A. Gunter, and Bo Li. IEEE S&P, 2021.

• Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs. [pdf] [code]

  • Soheil Kolouri, Aniruddha Saha, Hamed Pirsiavash, and Heiko Hoffmann. CVPR, 2020.

• One-Pixel Signature: Characterizing CNN Models for Backdoor Detection. [pdf]

  • Shanjiaoyang Huang, Weiqi Peng, Zhiwei Jia, and Zhuowen Tu. ECCV, 2020.

• Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases. [pdf] [code]

  • Ren Wang, Gaoyuan Zhang, Sijia Liu, Pin-Yu Chen, Jinjun Xiong, and Meng Wang. ECCV, 2020.

• Detecting Backdoor Attacks via Class Difference in Deep Neural Networks. [pdf]

  • Hyun Kwon. IEEE Access, 2020.

• Detecting Trojaned DNNs Using Counterfactual Attributions. [pdf]

  • Karan Sikka, Indranil Sur, Susmit Jha, Anirban Roy, and Ajay Divakaran. arXiv, 2020.

• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations. [pdf]

  • Xiaoyu Zhang, Ajmal Mian, Rohit Gupta, Nazanin Rahnavard, and Mubarak Shah. arXiv, 2020.

• Odyssey: Creation, Analysis and Detection of Trojan Models. [pdf] [dataset]

  • Marzieh Edraki, Nazmul Karim, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. arXiv, 2020.

• Noise-response Analysis for Rapid Detection of Backdoors in Deep Neural Networks. [pdf]

  • N. Benjamin Erichson, Dane Taylor, Qixuan Wu, and Michael W. Mahoney. arXiv, 2020.

• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations. [pdf]

  • Xijie Huang, Moustafa Alzantot, and Mani Srivastava. arXiv, 2019.

Poison Suppression based Empirical Defense

• Robust Anomaly Detection and Backdoor Attack Detection via Differential Privacy. [pdf] [code]

  • Min Du, Ruoxi Jia, and Dawn Song. ICLR, 2020.
    

• Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Trade-off. [pdf]

  • Eitan Borgnia, Valeriia Cherepanova, Liam Fowl, Amin Ghiasi, Jonas Geiping, Micah Goldblum, Tom Goldstein, and Arjun Gupta. ICASSP, 2021.

• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping. [pdf] [code]

  • Sanghyun Hong, Varun Chandrasekaran, Yiğitcan Kaya, Tudor Dumitraş, and Nicolas Papernot. arXiv, 2020.
    

• Removing Backdoor-Based Watermarks in Neural Networks with Limited Data. [pdf]

  • Xuankai Liu, Fengting Li, Bihan Wen, and Qi Li. arXiv, 2020.
    

Sample Filtering based Empirical Defense

• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection. [pdf]

  • Di Tang, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. USENIX Security, 2021.

• CLEANN: Accelerated Trojan Shield for Embedded Neural Networks. [pdf]

  • Mojan Javaheripi, Mohammad Samragh, Gregory Fields, Tara Javidi, and Farinaz Koushanfar. ICCAD, 2020.

• Robust Anomaly Detection and Backdoor Attack Detection via Differential Privacy. [pdf] [code]

  • Min Du, Ruoxi Jia, and Dawn Song. ICLR, 2020.
    

• SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems. [pdf]

  • Edward Chou, Florian Tramèr, and Giancarlo Pellegrino. IEEE S&P Workshop, 2020.

• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks. [pdf] [extension] [code]

  • Yansong Gao, Chang Xu, Derui Wang, Shiping Chen, Damith C. Ranasinghe, and Surya Nepal. ACSAC, 2019.

• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering. [pdf]

  • Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. AAAI Workshop, 2019.

• Deep Probabilistic Models to Detect Data Poisoning Attacks. [pdf]

  • Mahesh Subedar, Nilesh Ahuja, Ranganath Krishnan, Ibrahima J. Ndiour, and Omesh Tickoo. NeurIPS Workshop, 2019.
    

• Spectral Signatures in Backdoor Attacks. [pdf] [code]

  • Brandon Tran, Jerry Li, and Aleksander Madry. NeurIPS, 2018.
    

• Exposing Backdoors in Robust Machine Learning Models. [pdf]

  • Ezekiel Soremekun, Sakshi Udeshi, and Sudipta Chattopadhyay. arXiv, 2020.

• A Unified Framework for Analyzing and Detecting Malicious Examples of DNN Models. [pdf]

  • Kaidi Jin, Tianwei Zhang, Chao Shen, Yufei Chen, Ming Fan, Chenhao Lin, and Ting Liu. arXiv, 2020.

• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios. [pdf]

  • Hassan Ali, Surya Nepal, Salil S. Kanhere, and Sanjay Jha. arXiv, 2020.

• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks. [pdf]

  • Alvin Chan, and Yew-Soon Ong. arXiv, 2019.
    

Certificated Defense

• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing. [pdf]

  • Elan Rosenfeld, Ezra Winston, Pradeep Ravikumar, J. Zico Kolter. ICML, 2020.

• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing. [pdf]

  • Binghui Wang, Xiaoyu Cao, Jinyuan jia, and Neil Zhenqiang Gong. CVPR Workshop, 2020.

• RAB: Provable Robustness Against Backdoor Attacks. [pdf] [code]

  • Maurice Weber, Xiaojun Xu, Bojan Karlas, Ce Zhang, and Bo Li. arXiv, 2020.

Attack and Defense Towards Other Tasks and Paradigms

Natural Language Processing

• Weight Poisoning Attacks on Pre-trained Models. [pdf] [code]

  • Keita Kurita, Paul Michel, and Graham Neubig. ACL, 2020.

• A Backdoor Attack Against LSTM-based Text Classification Systems. [pdf]

  • Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. IEEE Access, 2019.

• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder. [pdf]

  • Alvin Chan, Yi Tay, Yew-Soon Ong, and Aston Zhang. EMNLP-Findings, 2020.

• Detecting Universal Trigger’s Adversarial Attack with Honeypot. [pdf]

  • Thai Le, Noseong Park, Dongwon Lee. arXiv, 2020.
    

• ONION: A Simple and Effective Defense Against Textual Backdoor Attacks. [pdf]

  • Fanchao Qi, Yangyi Chen, Mukai Li, Zhiyuan Liu, and Maosong Sun. arXiv, 2020.

• Mitigating Backdoor Attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification. [pdf]

  • Chuanshuai Chen, and Jiazhu Dai. arXiv, 2020.

• Trojaning Language Models for Fun and Profit. [pdf]

  • Xinyang Zhang, Zheng Zhang, and Ting Wang. arXiv, 2020.

• BadNL: Backdoor Attacks Against NLP Models. [pdf]

  • Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang. arXiv, 2020.

Graph Neural Networks

• Graph Backdoor. [pdf]

  • Zhaohan Xi, Ren Pang, Shouling Ji, and Ting Wang. arXiv, 2020.

• Backdoor Attacks to Graph Neural Networks. [pdf]

  • Zaixi Zhang, Jinyuan Jia, Binghui Wang, and Neil Zhenqiang Gong. NeurIPS Workshop, 2020.
    

Reinforcement Learning

• Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems. [pdf]

  • Yue Wang, Esha Sarkar, Michail Maniatakos, and Saif Eddin Jabari. arXiv, 2020.

• Trojdrl: Trojan attacks on deep reinforcement learning agents. [pdf]

  • Panagiota Kiourti, Kacper Wardega, Susmit Jha, and Wenchao Li. arXiv, 2019.

• Design of Intentional Backdoors in Sequential Models. [pdf]

  • Zhaoyuan Yang, Naresh Iyer, Johan Reimann, and Nurali Virani. arXiv, 2019.

Collaborative Learning

• How to Backdoor Federated Learning. [pdf]

  • Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. AISTATS, 2020 (arXiv, 2018).

• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning. [pdf]

  • Syed Zawad, Ahsan Ali, Pin-Yu Chen, Ali Anwar, Yi Zhou, Nathalie Baracaldo, Yuan Tian, and Feng Yan. AAAI, 2021.

• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning. [pdf]

  • Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. NeurIPS, 2020.

• DBA: Distributed Backdoor Attacks against Federated Learning. [pdf]

  • Chulin Xie, Keli Huang, Pinyu Chen, and Bo Li. ICLR, 2020.

• Defending Against Backdoors in Federated Learning with Robust Learning Rate. [pdf]

  • Mustafa Safa Ozdayi, Murat Kantarcioglu, and Yulia R. Gel. AAAI, 2021.

• The Limitations of Federated Learning in Sybil Settings. [pdf] [extension] [code]

  • Clement Fung, Chris J.M. Yoon, and Ivan Beschastnikh. RAID, 2020 (arXiv, 2018).

• Backdoor Attacks and Defenses in Feature-partitioned Collaborative Learning. [pdf]

  • Yang Liu, Zhihao Yi, and Tianjian Chen. ICML Workshop, 2020.

• Can You Really Backdoor Federated Learning? [pdf]

  • Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, and H. Brendan McMahan. NeurIPS Workshop, 2019.

• On Provable Backdoor Defense in Collaborative Learning. [pdf]

  • Ximing Qiao, Yuhua Bai, Siping Hu, Ang Li, Yiran Chen, and Hai Li. arXiv, 2021.

• Robust Federated Learning with Attack-Adaptive Aggregation. [pdf] [code]

  • Ching Pui Wan, and Qifeng Chen. arXiv, 2021.

• Meta Federated Learning. [pdf]

  • Omid Aramoon, Pin-Yu Chen, Gang Qu, and Yuan Tian. arXiv, 2021.

• FLGUARD: Secure and Private Federated Learning. [pdf]

  • Thien Duc Nguyen, Phillip Rieger, Hossein Yalame, Helen Möllering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, Azalia Mirhoseini, Ahmad-Reza Sadeghi, Thomas Schneider, and Shaza Zeitouni. arXiv, 2021.

• Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy. [pdf]

  • Mohammad Naseri, Jamie Hayes, and Emiliano De Cristofaro. arXiv, 2020.

• Backdoor Attacks on Federated Meta-Learning. [pdf]

  • Chien-Lun Chen, Leana Golubchik, and Marco Paolieri. arXiv, 2020.

• Dynamic backdoor attacks against federated learning. [pdf]

  • Anbu Huang. arXiv, 2020.

• Federated Learning in Adversarial Settings. [pdf]

  • Raouf Kerkouche, Gergely Ács, and Claude Castelluccia. arXiv, 2020.

• BlockFLA: Accountable Federated Learning via Hybrid Blockchain Architecture. [pdf]

  • Harsh Bimal Desai, Mustafa Safa Ozdayi, and Murat Kantarcioglu. arXiv, 2020.

• Mitigating Backdoor Attacks in Federated Learning. [pdf]

  • Chen Wu, Xian Yang, Sencun Zhu, and Prasenjit Mitra. arXiv, 2020.

• BaFFLe: Backdoor detection via Feedback-based Federated Learning. [pdf]

  • ebastien Andreina, Giorgia Azzurra Marson, Helen Möllering, and Ghassan Karame. arXiv, 2020.

• Learning to Detect Malicious Clients for Robust Federated Learning. [pdf]

  • Suyi Li, Yong Cheng, Wei Wang, Yang Liu, and Tianjian Chen. arXiv, 2020.

• Attack-Resistant Federated Learning with Residual-based Reweighting. [pdf] [code]

  • Shuhao Fu, Chulin Xie, Bo Li, and Qifeng Chen. arXiv, 2019.

Transfer Learning

• Backdoor Attacks against Transfer Learning with Pre-trained Deep Learning Models. [pdf]

  • Shuo Wang, Surya Nepal, Carsten Rudolph, Marthie Grobler, Shangyu Chen, and Tianle Chen. IEEE Transactions on Services Computing, 2020.

• Weight Poisoning Attacks on Pre-trained Models. [pdf] [code]

  • Keita Kurita, Paul Michel, and Graham Neubig. ACL, 2020.

• Latent Backdoor Attacks on Deep Neural Networks. [pdf]

  • Yuanshun Yao, Huiying Li, Haitao Zheng and Ben Y. Zhao. CCS, 2019.

• Red Alarm for Pre-trained Models: Universal Vulnerabilities by Neuron-Level Backdoor Attacks. [pdf] [code]

  • Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Xin Jiang, and Maosong Sun. arXiv, 2021.

Others

• Backdoor Attack against Speaker Verification [pdf] [code]

  • Tongqing Zhai, Yiming Li, Ziqi Zhang, Baoyuan Wu, Yong Jiang, and Shu-Tao Xia. ICASSP, 2021.

• NeuroAttack: Undermining Spiking Neural Networks Security through Externally Triggered Bit-Flips. [pdf]

  • Valerio Venceslai, Alberto Marchisio, Ihsen Alouani, Maurizio Martina, and Muhammad Shafique. IJCNN, 2020.

• Explainability Matters: Backdoor Attacks on Medical Imaging. [pdf]

  • Munachiso Nwadike, Takumi Miyawaki, Esha Sarkar, Michail Maniatakos, and Farah Shamout. AAAI Workshop, 2021.

• Trojan Attacks on Wireless Signal Classification with Adversarial Machine Learning. [pdf]

  • Kemal Davaslioglu, and Yalin E. Sagduyu. DySPAN, 2019.

• Backdoor Attacks on the DNN Interpretation System. [pdf]

  • Shihong Fang, and Anna Choromanska. NeurIPS Workshop, 2020.

• Embedding and Synthesis of Knowledge in Tree Ensemble Classifiers. [pdf]

  • Wei Huang, Xingyu Zhao, and Xiaowei Huang. arXiv, 2020.

• BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models. [pdf]

  • Ahmed Salem, Yannick Sautter, Michael Backes, Mathias Humbert, and Yang Zhang. arXiv, 2020.

• Targeted Forgetting and False Memory Formation in Continual Learners through Adversarial Backdoor Attacks. [pdf]

  • Muhammad Umer, Glenn Dawson, Robi Polikar. arXiv, 2020.

• Backdoors in Neural Models of Source Code. [pdf]

  • Goutham Ramakrishnan, and Aws Albarghouthi. arXiv, 2020.

• EEG-Based Brain-Computer Interfaces Are Vulnerable to Backdoor Attacks. [pdf]

  • Lubin Meng, Jian Huang, Zhigang Zeng, Xue Jiang, Shan Yu, Tzyy-Ping Jung, Chin-Teng Lin, Ricardo Chavarriaga, and Dongrui Wu. arXiv, 2020.

• Exploring Backdoor Poisoning Attacks Against Malware Classifiers. [pdf]

  • Giorgio Severi, Jim Meyer, Scott Coull, and Alina Oprea. arXiv, 2020.

• Bias Busters: Robustifying DL-based Lithographic Hotspot Detectors Against Backdooring Attacks. [pdf]

  • Kang Liu, Benjamin Tan, Gaurav Rajavendra Reddy, Siddharth Garg, Yiorgos Makris, and Ramesh Karri. arXiv, 2020.

Properties Discussion and Evaluation

• On the Trade-off between Adversarial and Backdoor Robustness. [pdf]

  • Cheng-Hsin Weng, Yan-Ting Lee, and Shan-Hung Wu. NeurIPS, 2020.

• A Tale of Evil Twins: Adversarial Inputs versus Poisoned Models. [pdf] [code]

  • Ren Pang, Hua Shen, Xinyang Zhang, Shouling Ji, Yevgeniy Vorobeychik, Xiapu Luo, Alex Liu, and Ting Wang. CCS, 2020.

• Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers. [pdf]

  • Loc Truong, Chace Jones, Brian Hutchinson, Andrew August, Brenda Praggastis, Robert Jasper, Nicole Nichols, and Aaron Tuor. CVPR Workshop, 2020.

• On Evaluating Neural Network Backdoor Defenses. [pdf]

  • Akshaj Veldanda, and Siddharth Garg. NeurIPS Workshop, 2020.

• Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks. [pdf] [code]

  • Avi Schwarzschild, Micah Goldblum, Arjun Gupta, John P Dickerson, and Tom Goldstein. NeurIPS Workshop, 2020.

• Rethinking the Trigger of Backdoor Attack. [pdf]

  • Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. arXiv, 2020.
    

• TROJANZOO: Everything You Ever Wanted to Know about Neural Backdoors (But were Afraid to Ask). [pdf] [code]

  • Ren Pang, Zheng Zhang, Xiangshan Gao, Zhaohan Xi, Shouling Ji, Peng Cheng, and Ting Wang. arXiv, 2020.

• Poisoned Classifiers are Not Only Backdoored, They are Fundamentally Broken. [pdf] [code]

  • Mingjie Sun, Siddhant Agarwal, and J. Zico Kolter. arXiv, 2020.

• Effect of Backdoor Attacks over the Complexity of the Latent Space Distribution. [pdf] [code]

  • Henry D. Chacon, and Paul Rad. arXiv, 2020.

• Trembling Triggers: Exploring the Sensitivity of Backdoors in DNN-based Face Recognition. [pdf]

  • Cecilia Pasquini, and Rainer Böhme. EURASIP Journal on Information Security, 2020.

• Backdoor Attacks on Facial Recognition in the Physical World. [pdf] [Master Thesis]

  • Emily Wenger, Josephine Passanati, Yuanshun Yao, Haitao Zheng, and Ben Y. Zhao. arXiv, 2020.

• Noise-response Analysis for Rapid Detection of Backdoors in Deep Neural Networks. [pdf]

  • N. Benjamin Erichson, Dane Taylor, Qixuan Wu, and Michael W. Mahoney. arXiv, 2020.

Backdoor Attack for Good

• Using Honeypots to Catch Adversarial Attacks on Neural Networks. [pdf]

  • Shawn Shan, Emily Wenger, Bolun Wang, Bo Li, Haitao Zheng, Ben Y. Zhao. CCS, 2020. (Note: Unfortunately, it was broken by Nicholas Carlini most recently. [arXiv])

• Turning Your Weakness into a Strength: Watermarking Deep Neural Networks by Backdooring. [pdf] [code]

  • Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. USENIX Security, 2018.

• Open-sourced Dataset Protection via Backdoor Watermarking. [pdf]

  • Yiming Li, Ziqi Zhang, Jiawang Bai, Baoyuan Wu, Yong Jiang, and Shu-Tao Xia. NeurIPS Workshop, 2020.

• What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space. [pdf]

  • Shihao Zhao, Xingjun Ma, Yisen Wang, James Bailey, Bo Li, and Yu-Gang Jiang. arXiv, 2021.

• What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors. [pdf]

  • Yi-Shan Lin, Wen-Chuan Lee, and Z. Berkay Celik. arXiv, 2020.

• Towards Probabilistic Verification of Machine Unlearning. [pdf] [code]

  • David Marco Sommer, Liwei Song, Sameer Wagh, and Prateek Mittal. arXiv, 2020.

Toolbox

• TrojanZoo

Competition

• IARPA TrojAI Competition