Abstract
While Variational Inference (VI) is central to modern generative models likeVariational Autoencoders (VAEs) and Denoising Diffusion Models (DDMs), itspedagogical treatment is split across disciplines. In statistics, VI istypically framed as a Bayesian method for posterior approximation. In machinelearning, however, VAEs and DDMs are developed from a Frequentist viewpoint,where VI is used to approximate a maximum likelihood estimator. This creates abarrier for statisticians, as the principles behind VAEs and DDMs are hard tocontextualize without a corresponding Frequentist introduction to VI. Thispaper provides that introduction: we explain the theory for VI, VAEs, and DDMsfrom a purely Frequentist perspective, starting with the classicalExpectation-Maximization (EM) algorithm. We show how VI arises as a scalablesolution for intractable E-steps and how VAEs and DDMs are natural,deep-learning-based extensions of this framework, thereby bridging the gapbetween classical statistical inference and modern generative AI.