Deep unsupervised learning

My notes from Berkeley's CS294-158 (Abbeel, Yan, Frans, Wu), one chapter per lecture, in the order the lectures took them. Two halves: building models of p(x) you can sample and evaluate, and inventing supervised problems out of unlabelled data so a network learns representations for free.

One of thirteen written so far. Entries in grey are lectures I have not got to yet — they go up as I work through them.

Foundations

• Motivation and the landscapeHinton's parameter budget as actual arithmetic, LeCun's cake with the bits attached, and compression as the definition of understanding

Likelihood-based models

• Autoregressive models
• Flow models
• Latent variable models and VAEs

Implicit and score-based models

• Generative adversarial networks
• Diffusion models

Learning without labels

• Self-supervised learning
• Semi-supervised learning
• Large language models

Applications

• Video generation
• Compression

Prerequisites: linear algebra, multivariable calculus, a first course in probability, and comfort with PyTorch. The information theory is built up from scratch in lecture one.