Probabilistic Latent Variable Inference Via Reparameterization Trick

via “scalable-posterior-inference-via-variational-approximation”

* 🏆 2006: [Reducing the Dimensionality of Data with Neural Networks (Autoencoder)](https://www.science.org/doi/abs/10.1126/science.1127647)

Unique: Introduces mean-field variational inference to topic modeling (Blei et al. 2003), replacing expensive Gibbs sampling with coordinate ascent optimization over variational parameters — enabling orders-of-magnitude speedup while maintaining interpretability through explicit posterior approximation

vs others: Dramatically faster than Gibbs sampling on large corpora (hours vs days) while providing explicit uncertainty estimates unlike deterministic LSA; trades some accuracy for scalability but remains more principled than heuristic approximations

* 🏆 2014: [Generative Adversarial Networks (GAN)](https://papers.nips.cc/paper/2014/hash/5ca3e9b122f61f8f06494c97b1afccf3-Abstract.html)

Unique: Introduces the reparameterization trick, which reformulates the variational objective to eliminate the need for score function estimators or other high-variance gradient approximations. This enables direct application of standard SGD to variational inference, whereas prior methods required specialized algorithms like REINFORCE or required discrete approximations. The key innovation is expressing the expectation over q(z|x) as a deterministic function of auxiliary noise variables, making the entire objective differentiable with respect to encoder parameters.

vs others: Scales to large datasets with continuous latents far more efficiently than classical variational inference methods (EM, mean-field approximation) because it avoids expensive E-step computations and uses mini-batch SGD; enables end-to-end neural network optimization unlike discrete latent variable models or non-differentiable inference schemes.