#noise-scale

Question 1: Single choice

A time-dependent U-Net score model for MNIST ends its forward pass with h = h / marginal_prob_std(t)[:, None, None, None], i.e. it divides its output by $\sigma_t$ . It is trained so that $\mathbf{s}_\theta(\mathbf{x} + \sigma_t\mathbf{z}, t) \approx -\mathbf{z}/\sigma_t$ . What is the main benefit and the main cost of this design?

The layers before the division only need to output about $-\mathbf{z}$ , of unit scale at every $t$ ; but errors are amplified when $\sigma_t$ is small1
It normalizes the input image, so the network sees pixel values in $[0, 1]$ at every noise level; but it slows down every training step2
It turns the score into a probability distribution whose entries sum to one; but it makes the sampler nondeterministic3
It removes the need for a time embedding, since $t$ now enters only through $\sigma_t$ ; but it doubles the parameter count4

Timed test