Deep Learning with Catalyst

This is an open deep learning course made by Deep Learning School, Tinkoff, and Catalyst team. Lectures and practice notebooks located in

./week*

./homework*

Note: the course is under update: weeks with colab barge are ready to go, weeks with [WIP] label are still in progress. You could use the

v20.12

branch for the earlier version of the full course.

Syllabus

• week 1: Deep learning intro
  • Deep learning – introduction, backpropagation algorithm. Optimization methods.
  • Neural Network in numpy.
• week 2: Deep learning frameworks
  • Regularization methods and deep learning frameworks.
  • Pytorch basics & extras.
• week 3: Convolutional Neural Network
  • CNN. Model Zoo.
  • Convolutional kernels. ResNet. Simple Noise Attack.
• week 4: Object Detection, Image Segmentation
  • Object Detection. (One, Two)-Stage methods. Anchors.
  • Image Segmentation. Up-scaling. FCN, U-net, FPN. DeepMask.
• week 5: Metric Learning
  • Metric Learning. Contrastive and Triplet Loss. Samplers.
  • Cross Entropy Loss modifications. SphereFace, CosFace, ArcFace.
• week 6: Autoencoders
  • AutoEncoders. Denoise, Sparse, Variational.
  • Generative Models. Autoregressive models.
• week 7: Generative Adversarial Models
  • Generative Adversarial Networks. VAE-GAN. AAE.
  • Energy based model.
• week 8: Natural Language Processing
  • Embeddings.
  • RNN. LSTM, GRU.
• week 9: Attention and transformer model
  • Attention Mechanism.
  • Transformer Model.
• week 10: Transfer Learning in NLP
  • Pretrained Transformers. BERT. GPT.
  • Data Augmentation in Texts. Domain Adaptation.
• week 11: Recommender Systems
  • Collaborative Filtering. FunkSVD.
  • Neural Collaborative Filtering.
• week 12: Reinforcement Learning for RecSys
  • DQN Algorithm.
  • DDPG Algorithm.
  • RecSim with Wolpertinger.
• [WIP] week 13: Extras
  • Research & Deploy.
  • Config API. Reaction.

Environment

Anaconda setup

# setup - env
conda create -n catalyst-dl python=3.7 anaconda
source activate catalyst-dl
conda remove nb_conda_kernels -y
conda install -c conda-forge nb_conda_kernels -y
conda install notebook jupyter nb_conda -y
conda remove nbpresent -y

setup - jupyter
jupyter notebook password
jupyter run
jupyter notebook --no-browser --ip 0.0.0.0 --port 8888

Requirements

pip install -U catalyst==21.04.2 torch==1.8.0 albumentations==0.5.0

Course staff & contributors

• @AlexeySh
• @artek0chumak
• @elephantmipt
• @Inkln
• @Fulldis
• @lordofprograms
• @Scitator
• @zelcookie