Reading List

Last updated on

There are too many books to read. In this section, I will list some main ideas of each book.

Reading path:

Table of Contents:

Books

Deep Learning Book

Dive into Deep Learning

Deep Learning with PyTorch - Eli Stevens, Luca Antiga

In-depth explainations of PyTorch operations and a complete project on image classification.

Highlights:

  • PyTorch memory management
  • Explore the image classification in depth over several chapters.

Contents:

  • Chapter 01: Introduction to PyTorch
  • Chapter 02: Pre-trained models in PyTorch
  • Chapter 03: PyTorch Tensors
    • PyTorch memory management
    • Indexing and operating on tensors
  • Chapter 04: Real-world data representation
  • Chapter 05: Mechanics of Learning
  • Chapter 06: Build a linear model
  • Chapter 07: Birds vs Airplanes
    • Datasets & DataLoaders
    • Classification loss
  • Chapter 08: build CNN
  • Chapter 09: LunaDataset. Load CT Data
  • Chapter 10: Classify Suspected Tumors
    • DataLoader
    • Conv3d, BatchNorm3d, LeakyReLU
    • 99.7% means done?
  • Chapter 11: Monitoring Metrics
    • Precision, Recall
    • TensorBoard
    • Data Augmentation

Programming PyTorch for Deep Learning - Ian Pointer

Practical PyTorch + Advanced techniques in 2019

Highlights

  • Many types of data: image, text, audio
  • New Debug Methods: Tensorboard, stacktrace
  • Advanced techniques: Mixup, smoothing, GAN, Transformer
  • Serving with Flask, Docker, Kubernetes

Contents

  • Chapter 01: Getting Started
    • Build PC vs Cloud
    • Tensor Operations, Tensor Boardcasting
  • Chapter 02: Image classification
  • Chapter 03: CNN
    • Pretrained
    • PyTorch Hub
  • Chapter 04: Transfer Learning
    • Transfer Learning with Resnet
    • Differential Learning Rates
    • Data Agumentation
    • Ensembles
  • Chapter 05: Text Classification
    • RNN, LSTM, biLSTM
    • Embedding
    • torchtext
    • Data Augmentation
  • Chapter 06: Sound
    • ESC-50 Dataset
    • CNN Model for ESC-60
    • Spectrogram, Wide ResNet
    • Audio Data Augmentation, torchaudio
  • Chapter 07: Debugging PyTorch Models
    • Tensorboard, PyTorch hooks
    • Flame Graphs: stacktraces to debug performance
    • Debug GPU
  • Chapter 08: PyTorch in Production
    • Model Serving with Flask + Docker + Telemetry
    • Kubernetes
    • TorchSript
    • libTorch
  • Chapter 09: PyTorch in the Wild
    • Data Augmentation: MixUP and Label Smoothing
    • Super Resolution + GAN + ESRGAN
    • Adversarial Samples
    • pytorch-transformers: Transformer Architecture: BERT, GPT-2, ULMFiT

Deep Learning for Computer Vision

Hands-on Machine Learning with Scikit-Learn Keras and TensorFlow 2.0

Best hands-on book on Machine Learning with TensorFlow 2.0

Highlights

  • Describe various machine learning & deep learning techniques
  • Support Keras & TensorFlow 2.0

Contents

  • Chapter 01: Machine Learning landscape
  • Chapter 02: End-to-End ML Project
  • Chapter 03: Classification
    • Cross-Validation & Confusion Matrix
    • Precision, Recall, ROC Curve
  • Chapter 04: Training models
  • Chapter 05: SVM
  • Chapter 06: Decision Trees
  • Chapter 07: Ensemble, Random forests
  • Chapter 08: PCA
  • Chapter 09: Unsupervised Techniques
  • Chapter 10: Neural Networks with Keras
  • Chapter 11: Training Deep networks
    • Vanishing/Exploding Gradients: He init, activation, batch norm, gradient clipping
    • Pretrained Layers
    • Faster Optimizers
    • Regularization: l1, l2, droupout, max-norm
  • Chapter 12: Custom models with Tensorflow
    • Use Tensorflow to extend keras.models, …
  • Chapter 13: Tensorflow Data
    • Data API, TFRecord, Features API
    • TF Transform
  • Chapter 14: CNN
    • CNN Architectures
    • ResNet with Keras
  • Chapter 15: Sequences with RNN & CNN
    • RNN
    • Forecasting
    • Long Sequences
    • Wavenet
  • Chapter 16: NLP With RNN and Attention
    • Char-RNN
    • Sentiment analysis
    • Bidirectional RNNs & Beam search
    • Attenion machanisms
  • Chapter 17: Autoencoders & GAN
    • Autoencoders: Convolutional, recurrent, denoising, sparse, variational
    • GAN: Deep Convolutional GAN, Progressive Growing, StyleGANs
  • Chapter 18: RL
  • Chapter 19: Training & Deploying
    • Serving on GCP
    • Mobile / Embedded
    • Distributed training with Tensorflow Cluster

Deep Learning from Scratch - Seth Weidman

This book shows how to implement neural network techniques in numpy.

Highlights:

  • Explains in math -> diagram -> code.
  • Code forward and backward pass of CNN and RNN.
  • Appendix: Detailed matrix derivatives + code

Contents:

  • Chapter 01: Foundations
    • Functions + Derivatives + Backward pass
  • Chapter 02: Fundamentals
    • Explain Supervised Learning
    • Code Linear Regression with gradient descent.
  • Chapter 03: Deep Learning from scratch
    • Build Operation base class
    • Build classes: Layers -> Network -> Optimizer -> Trainer
  • Chapter 04: Extensions
    • Softmax cross entropy loss
    • Momentum, learning rate decay
    • Weight initialization
    • Dropout
  • Chapter 05: CNN
    • Code Conv2D layer with forward & backward pass
  • Chapter 06: RNN
    • Code gradient accumulation
    • Code RNNLayer -> Vanilla -> GRU -> LSTM
  • Chapter 07: PyTorch
  • Appendix:
    • Matrix chain rule
    • Gradient w.r.t bias
    • Convolutions via matrix multiplication

Grokking Deep Learning - Andrew W. Trask

Another Deep Learning from sratch with numpy

Highlights

  • Code Neural networks operations, layers with numpy
  • Build framework with autograd and support lots of layer types
  • Federated learning

Contents

  • Chapter 01: Why learn deep learning
  • Chapter 02: Fundamentals
    • Supervised, unsupervised, nonparameteric
  • Chapter 03: Forward Propagation
  • Chapter 04: Gradient Descent
  • Chapter 05: Generalize Gradient Descent
  • Chapter 06: Backpropagation
  • Chapter 07: Visualize Neural Network
  • Chapter 08: Regularization & Batching
  • Chapter 09: Activation functions
  • Chapter 10: Convolution
  • Chapter 11: Word embeddings
  • Chapter 12: Recurrent layers
  • Chapter 13: Autograd framework
  • Chapter 14: LSTM
  • Chapter 15: Federated learning
  • Chapter 16: where to go?

Interpretable Machine Learning

Deep Learning Illustrated

GANs in Action

Math for Programmers

Avatar
Quan Hua
Deep Learning Enthusiast
comments powered by Disqus