#model-capacity

Question 1: Single choice

Why does common practice prefer a large, regularized network over a small network that is unable to overfit?

Regularization can only reduce effective complexity; it cannot help a model too small to describe the data1
Small networks train more slowly, because each of their parameters receives a larger gradient2
Regularization adds parameters, so it turns a small network into a large one3
Large networks never overfit when they are trained with SGD4