#function-classes

Question 1: Single choice

With residual connections, each larger ResNet contains the smaller ones as special cases: the function classes are nested. Why can we still not keep adding layers and expect ever better test accuracy?

Nesting only guarantees an equal or better training fit; a bigger class can overfit and costs more1
Every added layer raises the training error, because the identity path accumulates noise from all earlier layers2
Beyond about 152 layers the residual branches can no longer learn the zero mapping, so the classes stop being nested3
Nested classes all contain exactly the same functions, so extra layers add no expressive power4

Timed test