#prediction-mode

Question 1: Select all that apply

Suppose a nonlinearity normalizes each hidden unit using the mean and variance of the current minibatch, as batch normalization does. Which problems does this cause? Select all that apply.

With minibatches of size 1, every normalized activation is 0, so nothing can be learned1
Prediction needs fixed statistics, such as averages from training, so outputs do not depend on the batch2
Gradients can no longer be computed by backpropagation3
The network can no longer be trained with minibatch SGD4
An example's output depends on which other examples share its minibatch5

Timed test