April 2026 Course -Lecture 3: Before DDPM: Thermodynamics and Score Matching

1538 pages of documents. No vids.

Core papers:

1. Deep Unsupervised Learning using Nonequilibrium Thermodynamics
   Jascha Sohl-Dickstein et al., 2015

2. Generative Modeling by Estimating Gradients of the Data Distribution
   Yang Song, Stefano Ermon, 2019

Topics:

• Forward diffusion

• Reverse diffusion

• Markov chains

• Score matching

• Why “adding noise and learning to reverse it” is mathematically grounded

Key concept:

data → gradually add noise → Gaussian noise
Gaussian noise → learned reverse process → data

Engineering connection:

• Why diffusion training is stable
  

• Why predicting denoising directions can learn the data distribution