Practical Deep Learning for Coders - fast.ai

Teaches deep learning by starting with high-level applications (image classification, NLP) and progressively revealing underlying mathematics and theory, rather than bottom-up linear algebra foundations. Uses Jupyter notebooks embedded in the course platform to interleave video lectures, code examples, and interactive exercises in a single learning context. The curriculum is structured around real datasets and competitions (ImageNet, MNIST variants) to anchor abstract concepts in concrete problems.

Teaches and provides code patterns for leveraging pre-trained convolutional neural networks (ResNet, EfficientNet, Vision Transformers) trained on ImageNet, then fine-tuning only the final layers on custom datasets with as few as 10-100 images per class. The fastai library implements discriminative learning rates (lower learning rates for early layers, higher for later layers) and progressive unfreezing to stabilize training on small datasets. Includes techniques like data augmentation and learning rate scheduling to prevent overfitting.

Teaches best practices for creating high-quality training datasets, including data collection strategies, annotation guidelines, and quality control. Covers how to use annotation tools (LabelImg, CVAT, Prodigy), manage annotation workflows with multiple annotators, and measure inter-annotator agreement. Discusses the importance of dataset diversity, handling class imbalance, and avoiding common pitfalls like data leakage. Includes practical guidance on data augmentation to increase effective dataset size.

Teaches how to select learning rates and other hyperparameters to train deep learning models effectively. Covers the learning rate finder (plotting loss vs learning rate to identify optimal ranges), learning rate schedules (constant, step decay, cosine annealing), and momentum/weight decay tuning. Includes techniques like discriminative learning rates (different rates for different layers) and cyclical learning rates. Discusses the relationship between batch size, learning rate, and convergence speed.

Teaches NLP using transfer learning with pre-trained language models (ULMFiT, BERT-style architectures) for tasks like text classification, sentiment analysis, and named entity recognition. The course covers the Universal Language Model Fine-tuning (ULMFiT) approach: pre-train on general text corpus, fine-tune on task-specific corpus, then fine-tune on labeled data. Includes practical patterns for handling variable-length sequences, building custom tokenizers, and interpreting model predictions via attention weights.

Teaches recommendation systems using collaborative filtering, specifically matrix factorization with embeddings. The approach learns latent representations for users and items by factorizing the user-item interaction matrix, then predicts ratings or rankings by computing dot products of learned embeddings. The course covers both explicit feedback (ratings) and implicit feedback (clicks, purchases), regularization techniques to prevent overfitting, and how to handle cold-start problems with content-based fallbacks.

Teaches how to apply deep learning to tabular/structured data (CSV files with mixed categorical and continuous features) using entity embeddings and shallow neural networks. The approach learns dense vector representations for categorical variables (like country, product category) rather than one-hot encoding, then concatenates embeddings with continuous features and passes through a small MLP. Includes techniques for handling missing values, feature scaling, and regularization via dropout and batch normalization.

Teaches generative deep learning using Generative Adversarial Networks (GANs) and diffusion models for image synthesis. Covers the adversarial training loop (generator vs discriminator), loss functions (Wasserstein, spectral normalization), and practical stabilization techniques. Includes applications like style transfer, super-resolution, and image-to-image translation. The course explains how diffusion models iteratively denoise random noise to generate images, contrasting with GAN training dynamics.

Teaches object detection (localizing and classifying multiple objects in an image) and instance segmentation (pixel-level masks for each object) using architectures like Faster R-CNN, RetinaNet, and Mask R-CNN. Covers the two-stage detection pipeline (region proposal network + classification/localization head), anchor-based vs anchor-free approaches, and loss functions (focal loss for class imbalance). Includes practical patterns for handling variable image sizes, non-maximum suppression, and evaluation metrics (mAP).

Teaches time-series prediction using recurrent neural networks (LSTMs, GRUs) and attention mechanisms (Transformers). Covers sequence-to-sequence models for multi-step forecasting, handling variable-length sequences, and techniques like teacher forcing during training. Includes practical patterns for feature engineering (lag features, rolling statistics), handling missing values, and evaluating forecasts with metrics like RMSE and MAE. Discusses the trade-off between model complexity and interpretability.

Teaches techniques for understanding what a trained deep learning model has learned and which features drive predictions. Covers gradient-based methods (saliency maps, integrated gradients), attention visualization, permutation importance, and SHAP values. For images, includes techniques like Class Activation Maps (CAM) and Grad-CAM to visualize which regions influence predictions. Emphasizes the importance of model interpretability for debugging, building trust, and meeting regulatory requirements.

Teaches how to take a trained fastai model and deploy it for inference in production environments. Covers model export (to ONNX, TorchScript), quantization (int8, float16) for reducing model size and latency, and deployment patterns (REST API, serverless functions, edge devices). Includes practical guidance on handling batch inference, caching, and monitoring model performance in production. Discusses the trade-off between model accuracy and inference speed.