Need help with deeplearning-models?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

rasbt
14.4K Stars 3.6K Forks MIT License 102 Commits 2 Opened issues

Description

A collection of various deep learning architectures, models, and tips

Services available

!
?

Need anything else?

Contributors list

# 237
Jupyter...
scikit-...
TeX
logisti...
61 commits
# 18,034
Python
calculu...
machine...
Jupyter...
2 commits
# 62,003
Jupyter...
Python
1 commit
# 61,963
Jupyter...
Python
1 commit
# 60,238
Python
Open Da...
data-sc...
Shell
1 commit
# 10,220
Jupyter...
Scala
Azure
pyspark
1 commit
# 61,970
Jupyter...
Python
1 commit
# 58,541
Jupyter...
Python
regular...
pytorch
1 commit

Python 3.7

Deep Learning Models

A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.

Traditional Machine Learning

Multilayer Perceptrons

Convolutional Neural Networks

Basic

Concepts

  • Replacing Fully-Connnected by Equivalent Convolutional Layers
       [PyTorch: GitHub | Nbviewer]

AlexNet

DenseNet

  • DenseNet-121 Digit Classifier Trained on MNIST
       [PyTorch: GitHub | Nbviewer]
  • DenseNet-121 Image Classifier Trained on CIFAR-10
       [PyTorch: GitHub | Nbviewer]

Fully Convolutional

  • Fully Convolutional Neural Network
       [PyTorch: GitHub | Nbviewer]

LeNet

MobileNet

Network in Network

  • Network in Network CIFAR-10 Classifier
       [PyTorch: GitHub | Nbviewer]

VGG

ResNet

  • ResNet and Residual Blocks
       [PyTorch: GitHub | Nbviewer]
  • ResNet-18 Digit Classifier Trained on MNIST
       [PyTorch: GitHub | Nbviewer]
  • ResNet-18 Gender Classifier Trained on CelebA
       [PyTorch: GitHub | Nbviewer]
  • ResNet-34 Digit Classifier Trained on MNIST
       [PyTorch: GitHub | Nbviewer]
  • ResNet-34 Object Classifier Trained on QuickDraw
       [PyTorch: GitHub | Nbviewer]
  • ResNet-34 Gender Classifier Trained on CelebA
       [PyTorch: GitHub | Nbviewer]
  • ResNet-50 Digit Classifier Trained on MNIST
       [PyTorch: GitHub | Nbviewer]
  • ResNet-50 Gender Classifier Trained on CelebA
       [PyTorch: GitHub | Nbviewer]
  • ResNet-101 Gender Classifier Trained on CelebA
       [PyTorch: GitHub | Nbviewer]
  • ResNet-101 Trained on CIFAR-10
       [PyTorch: GitHub | Nbviewer]
  • ResNet-152 Gender Classifier Trained on CelebA
       [PyTorch: GitHub | Nbviewer]

Normalization Layers

  • BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier
       [PyTorch: GitHub | Nbviewer]
  • Filter Response Normalization for Network-in-Network CIFAR-10 Classifier
       [PyTorch: GitHub | Nbviewer]

Metric Learning

  • Siamese Network with Multilayer Perceptrons
       [TensorFlow 1: GitHub | Nbviewer]

Autoencoders

Fully-connected Autoencoders

Convolutional Autoencoders

  • Convolutional Autoencoder with Deconvolutions / Transposed Convolutions
       [TensorFlow 1: GitHub | Nbviewer]
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Autoencoder with Deconvolutions (without pooling operations)
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Autoencoder with Nearest-neighbor Interpolation
       [TensorFlow 1: GitHub | Nbviewer]
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw
       [PyTorch: GitHub | Nbviewer]

Variational Autoencoders

Conditional Variational Autoencoders

  • Conditional Variational Autoencoder (with labels in reconstruction loss)
       [PyTorch: GitHub | Nbviewer]
  • Conditional Variational Autoencoder (without labels in reconstruction loss)
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss)
       [PyTorch: GitHub | Nbviewer]
  • Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss)
       [PyTorch: GitHub | Nbviewer]

Generative Adversarial Networks (GANs)

Graph Neural Networks (GNNs)

  • Most Basic Graph Neural Network with Gaussian Filter on MNIST
       [PyTorch: GitHub | Nbviewer]
  • Basic Graph Neural Network with Edge Prediction on MNIST
       [PyTorch: GitHub | Nbviewer]
  • Basic Graph Neural Network with Spectral Graph Convolution on MNIST
       [PyTorch: GitHub | Nbviewer]

Recurrent Neural Networks (RNNs)

Many-to-one: Sentiment Analysis / Classification

  • A simple single-layer RNN (IMDB)
       [PyTorch: GitHub | Nbviewer]
  • A simple single-layer RNN with packed sequences to ignore padding characters (IMDB)
       [PyTorch: GitHub | Nbviewer]
  • RNN with LSTM cells (IMDB)
       [PyTorch: GitHub | Nbviewer]
  • RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors
       [PyTorch: GitHub | Nbviewer]
  • RNN with LSTM cells and Own Dataset in CSV Format (IMDB)
       [PyTorch: GitHub | Nbviewer]
  • RNN with GRU cells (IMDB)
       [PyTorch: GitHub | Nbviewer]
  • Multilayer bi-directional RNN (IMDB)
       [PyTorch: GitHub | Nbviewer]
  • Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News)
       [PyTorch: GitHub | Nbviewer]

Many-to-Many / Sequence-to-Sequence

  • A simple character RNN to generate new text (Charles Dickens)
       [PyTorch: GitHub | Nbviewer]

Ordinal Regression

  • Ordinal Regression CNN -- CORAL w. ResNet34 on AFAD-Lite
       [PyTorch: GitHub | Nbviewer]
  • Ordinal Regression CNN -- Niu et al. 2016 w. ResNet34 on AFAD-Lite
       [PyTorch: GitHub | Nbviewer]
  • Ordinal Regression CNN -- Beckham and Pal 2016 w. ResNet34 on AFAD-Lite
       [PyTorch: GitHub | Nbviewer]

Tips and Tricks

  • Cyclical Learning Rate
       [PyTorch: GitHub | Nbviewer]
  • Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet)
       [PyTorch: GitHub | Nbviewer]
  • Gradient Clipping (w. MLP on MNIST)
       [PyTorch: GitHub | Nbviewer]

Transfer Learning

  • Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10)
       [PyTorch: GitHub | Nbviewer]

Visualization and Interpretation

  • Vanilla Loss Gradient (wrt Inputs) Visualization (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images)
       [PyTorch: GitHub | Nbviewer]
  • Guided Backpropagation (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images)
       [PyTorch: GitHub | Nbviewer]

PyTorch Workflows and Mechanics

Custom Datasets

  • Custom Data Loader Example for PNG Files
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD)
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images
       [PyTorch: GitHub | Nbviewer]
  • Using PyTorch Dataset Loading Utilities for Custom Datasets -- Fashion MNIST
       [PyTorch: GitHub | Nbviewer]

Training and Preprocessing

Improving Memory Efficiency

  • Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10)
       [PyTorch: GitHub | Nbviewer]

Parallel Computing

  • Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA
       [PyTorch: GitHub | Nbviewer]
  • Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example)    [PyTorch: GitHub | Nbviewer]

Other

  • PyTorch with and without Deterministic Behavior -- Runtime Benchmark
       [PyTorch: GitHub | Nbviewer]
  • Sequential API and hooks
       [PyTorch: GitHub | Nbviewer]
  • Weight Sharing Within a Layer
       [PyTorch: GitHub | Nbviewer]
  • Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib
       [PyTorch: GitHub | Nbviewer]

Autograd

  • Getting Gradients of an Intermediate Variable in PyTorch
       [PyTorch: GitHub | Nbviewer]

TensorFlow Workflows and Mechanics

Custom Datasets

  • Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives
       [TensorFlow 1: GitHub | Nbviewer]
  • Storing an Image Dataset for Minibatch Training using HDF5
       [TensorFlow 1: GitHub | Nbviewer]
  • Using Input Pipelines to Read Data from TFRecords Files
       [TensorFlow 1: GitHub | Nbviewer]
  • Using Queue Runners to Feed Images Directly from Disk
       [TensorFlow 1: GitHub | Nbviewer]
  • Using TensorFlow's Dataset API
       [TensorFlow 1: GitHub | Nbviewer]

Training and Preprocessing

  • Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives
       [TensorFlow 1: GitHub | Nbviewer]

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.