Score Matching Training Via Noise Prediction

via “noise prediction loss computation for diffusion training”

Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

Unique: Implements noise prediction loss by sampling random diffusion steps and computing L2 distance between U-Net predictions and ground-truth added noise, enabling efficient training without unrolling the full diffusion process

vs others: More computationally efficient than unrolled diffusion training; provides stable gradients compared to some alternative objectives, though equal step weighting may not optimize perceptual quality

via “score-matching-training-via-noise-prediction”

* 🏆 2020: [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)](https://arxiv.org/abs/2010.11929)

Unique: DDPM's training objective is derived from score-matching, where the score function (gradient of log probability) is approximated by predicting the noise added at each timestep. This connection provides theoretical grounding in score-based generative modeling and enables efficient training. The approach is more principled than VAE objectives and more stable than GAN training.

vs others: More theoretically grounded than VAE objectives, more stable than GAN training, and enables flexible noise weighting for improved sample quality.