Reverse Diffusion Sampling Algorithm Explanation

via “reverse-diffusion-sampling-with-learned-variance”

* 🏆 2020: [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)](https://arxiv.org/abs/2010.11929)

Unique: DDPM's reverse process is derived mathematically from the forward process, enabling principled sampling without requiring a separate decoder or post-processing. The variance can be fixed (using forward process variance) or learned, with learned variance often providing marginal improvements at added complexity. The sampling procedure is simple: iteratively apply the learned mean and add Gaussian noise until reaching t=0.

vs others: More stable and controllable than GAN sampling (no mode collapse, explicit noise control), higher quality than VAE decoding at comparable model size, and enables fine-grained quality-speed tradeoffs via step reduction.

 

Unique: Explicitly connects the reverse process to score-based generative modeling and provides side-by-side implementations of DDPM (full timesteps) vs DDIM (accelerated sampling), showing architectural differences in how timesteps are scheduled

vs others: More pedagogically structured than research papers, with runnable code examples that show both the mathematical theory and practical implementation details of sampling algorithms