Learn Deep Learning with AI Summer

A collection of all articles (almost 100) written for the AI Summer blog organized by topic.

Deep Learning Theory

Machine Learning

• A journey into Optimization algorithms for Deep Neural Networks
• Regularization techniques for training deep neural networks
• In-layer normalization techniques for training very deep neural networks
• Explainable AI (XAI): A survey of recents methods, applications and frameworks
• Spiking Neural Networks: where neuroscience meets artificial intelligenc

Convolutional Neural Networks

• Best deep CNN architectures and their principles: from AlexNet to EfficientNet
• Intuitive Explanation of Skip Connections in Deep Learning
• Understanding the receptive field of deep convolutional networks

Recurrent Neural Networks

• Recurrent neural networks: building a custom LSTM cell (colab / repo)
• Recurrent Neural Networks: building GRU cells VS LSTM cells in Pytorch
• Predict Bitcoin price with Long sort term memory Networks (LSTM)

Autoencoders

• How to Generate Images using Autoencoders
• The theory behind Latent Variable Models: formulating a Variational Autoencoder

Generative Adversarial Networks

• Decrypt Generative Adversarial Networks (GAN)
• GANs in computer vision - Introduction to generative learning (repo)
• GANs in computer vision - Conditional image synthesis and 3D object generation
• GANs in computer vision - Improved training with Wasserstein distance, game theory control and progressively growing schemes
• GANs in computer vision - 2K image and video synthesis, and large-scale class-conditional image generation
• GANs in computer vision - self-supervised adversarial training and high-resolution image synthesis with style incorporation
• GANs in computer vision - semantic image synthesis and learning a generative model from a single image

Attention and Transformers

• How Attention works in Deep Learning: understanding the attention mechanism in sequence models
• How Transformers work in deep learning and NLP: an intuitive introduction
• How the Vision Transformer (ViT) works in 10 minutes: an image is worth 16x16 words
• How Positional Embeddings work in Self-Attention (code in Pytorch)
• Why multi-head self attention works: math, intuitions and 10+1 hidden insights
• Transformers in computer vision: ViT architectures, tips, tricks and improvements

Graph Neural Networks

• Graph Neural Networks - An overview
• How Graph Neural Networks (GNN) work: introduction to graph convolutions from scratch (colab)
• Best Graph Neural Network architectures: GCN, GAT, MPNN and more

Self-supervised Learning

• Grokking self-supervised (representation) learning: how it works in computer vision and why
• Self-supervised representation learning on videos
• Understanding SWAV: self-supervised learning with contrasting cluster assignments

Reinforcement Learning

• The secrets behind Reinforcement Learning
• Deep Q Learning and Deep Q Networks
• Q-targets, Double DQN and Dueling DQN
• Unravel Policy Gradients and REINFORCE
• The idea behind Actor-Critics and how A2C and A3C improve them
• Trust Region and Proximal policy optimization (TRPO and PPO)

Deep Learning Applications

Image segmentation

• An overview of Unet architectures for semantic segmentation and biomedical image segmentation
• Semantic Segmentation in the era of Neural Networks

Object detection

• Localization and Object Detection with Deep Learning
• YOLO - You only look once (Single shot detectors)

Human pose estimation

• Human Pose Estimation

Face synthesis

• A closer look on Deepfakes: face sythesis with StyleGAN, face swap with XceptionNet and facial attributes and expression manipulation with StarGAN

Recommendation systems

• An introduction to Recommendation Systems: an overview of machine and deep learning architectures

Autonomous cars

• Self-driving cars using Deep Learning

Medical imaging

• Deep learning in medical imaging - 3D medical image segmentation with PyTorch (repo)
• Understanding coordinate systems and DICOM for deep learning medical image analysis
• Introduction to 3D medical imaging for machine learning: preprocessing and augmentations (colab)
• Deep learning in MRI beyond segmentation: Medical image reconstruction, registration, and synthesis
• Transfer learning in medical imaging: classification and segmentation
• Introduction to medical image processing with Python: CT lung and vessel segmentation without labels (colab / repo )
• 3D Medical image segmentation with transformers tutorial (colab)

Audio

• Speech synthesis: A review of the best text to speech architectures with Deep Learning
• Speech Recognition: a review of the different deep learning approaches

Biology

• Deep learning on computational biology and bioinformatics tutorial: from DNA to protein folding and alphafold2

MLOps (repo)

Best practices

• Deep Learning in Production: Laptop set up and system design
• Best practices to write Deep Learning code: Project structure, OOP, Type checking and documentation
• How to Unit Test Deep Learning: Tests in TensorFlow, mocking and test coverage
• Logging and Debugging in Machine Learning - How to use Python debugger and the logging module to find errors in your AI application

Data Processing

• Data preprocessing for deep learning: How to build an efficient big data pipeline
• Data preprocessing for deep learning: Tips and tricks to optimize your data pipeline using Tensorflow

Training

• How to build a custom production-ready Deep Learning Training loop in Tensorflow from scratch
• How to train a deep learning model in the cloud
• Distributed Deep Learning training: Model and Data Parallelism in Tensorflow

Deployment

• Deploy a Deep Learning model as a web application using Flask and Tensorflow
• How to use uWSGI and Nginx to serve a Deep Learning model
• How to use Docker containers and Docker Compose for Deep Learning applications
• Scalability in Machine Learning: Grow your model to serve millions of users
• Introduction to Kubernetes with Google Cloud: Deploy your Deep Learning model effortlessly

Libraries tutorials

JAX (repo)

• JAX for Machine Learning: how it works and why learn it
• Build a Transformer in JAX from scratch: how to write and train your own models
• JAX vs Tensorflow vs Pytorch: Building a Variational Autoencoder (VAE)

Tensorflow Extended (TFX)

• Tensorflow Extended (TFX) in action: build a production ready deep learning pipeline

Hugging face

• A complete Hugging Face tutorial: how to build and train a vision transformer (repo)

Weights and Biases (wandb)

• A complete Weights and Biases tutorial

Einsum and einops

• Understanding einsum for Deep learning: implement a transformer with multi-head self-attention from scratch (repo)

GPU programming

• Neural Network from scratch-part 1
• Neural Network from scratch-part 2

Miscellaneous

• Document clustering
• Explain Neural Arithmetic Logic Units (NALU)
• How to get hired as a Machine Learning Engineer
• Apply Machine Learning to your Business

Resources

• Top 10 courses to learn Machine and Deep Learning
• Best Artificial Intelligence books to read
• The Best Machine Learning books to learn AI
• Best bootcamps and programs to learn Machine Learning and Data Science
• Best Resources to Learn Deep Learning Theory
• Top Resources to start with Computer Vision and Deep Learning
• Best AI and Deep learning books to read in 2022