When a diffusion model is not memorizing the training samples, what does it generate, and why? In this talk, I will describe a quantitative framework for understanding the distribution produced by a learned diffusion model through a data-driven geometric object: a log-density ridge manifold of the smoothed training distribution. This manifold acts as a backbone for generation and reveals a three-stage inference behavior: trajectories first reach the ridge, then align in normal directions, and finally slide along tangent directions. 

This perspective allows us to quantify how training error influences generation in different directions, and to explain when inter-mode generations arise. I will also present a random feature example in which the model’s inductive bias can be decomposed explicitly into architectural bias and optimization error, and tracked along the inference dynamics. Experiments on synthetic multimodal distributions and MNIST latent diffusion support the theory in both low- and high-dimensional settings.