Deep generative models

My notes from IISc's E1 286s (Prof. Prathosh A. P.), rewritten for me rather than for anyone else. Everything here reduces to one problem: estimate an unknown distribution from finite samples, then learn to sample from it. The order below is the order the lectures took, because re-reading it should feel like sitting through them again.

Two things I try to hold to. Explain the intuition first — how I could have arrived at the result myself — and then make the derivation match that intuition instead of taking the slicker route. Entries in grey are notes I have not written yet.

Foundations

• Why we need probabilitywhere deterministic functions run out, and the first bricks: measures, sample spaces, random variables
• Prediction, sampling and divergence minimisationthe three-step recipe, and KL out of Shannon's axioms

Adversarial models

• f-divergencesone formula for KL, JS and TV, and the convex-conjugate trick that makes them estimable from samples
• From f-divergences to a trainable GANFenchel duality, the optimal discriminator, and where the log 4 comes from
• The classifier view, conditional GANs and CycleGANmode collapse as a stationary point; DCGAN, conditioning, the round trip
• Wasserstein GANs and optimal transportwhy Jensen-Shannon goes flat, and what the Lipschitz constraint costs — also domain adaptation and the first latent variable models
• Spectral normalisation
• Evaluating generative modelsFID, IS, and why both are unsatisfying

Latent variable models

• Variational autoencodersthe ELBO's two halves, and the reparameterisation trick measured against the score-function estimator
• β-VAE and VQ-VAEwhat the two halves are for once training ends; the codebook and its biased gradient
• Posterior collapsepulled out of ch-07 and ch-08, where it appears twice for different reasons
• Hierarchical VAEs

Diffusion

• Denoising diffusion probabilistic modelsa VAE with three restrictions imposed, each of which removes a difficulty
• Score matching, guidance and DDIMTweedie's formula, conditioning as one line of Bayes, and a fifty-fold speed-up for free
• Flow matching
• Latent diffusion and the modern architectures

Exact likelihood

• Normalizing flowsone conservation law, then the search for invertible layers whose Jacobian determinant you can read off
• Autoregressive modelsPixelCNN, and the tradeoff flows were trying to escape

Reference

• Concept mapwhat each lecture covers and where the series is headed
• Notation tableone place for every symbol the series reuses

Prerequisites, roughly: linear algebra, basic calculus, a first course in probability, and enough Python to read a training loop. Chapters 2, 3 and 5–6 of Goodfellow et al. cover it.