#training-objective

Question 1: Single choice

Denoising score matching trains a score network by minimizing $\mathbb{E}\,\lVert \sigma_t\,\mathbf{s}_\theta(\mathbf{x} + \sigma_t \mathbf{z}, t) + \mathbf{z}\rVert^2$ over clean training points $\mathbf{x}$ and Gaussian noise $\mathbf{z}$ . With two or more distinct training points, can a flexible enough network drive this loss to zero?

No: one noisy input can arise from different clean points with different noise, so the best output is an average1
Yes: a network with enough capacity can memorize the noise that was added to every training example2
Yes, but only if every training example is noised with the same noise level $\sigma_t$ 3
No, because the loss always contains the term $\lVert\mathbf{z}\rVert^2$ , which does not depend on $\theta$ 4

Timed test