#identity-mapping

Question 1: Single choice

A residual block computes $\mathbf{y} = \mathbf{x} + F(\mathbf{x})$ , where $F$ is a small stack of weight layers and the $+$ is the shortcut connection. What happens when the weights and biases of the last layer of $F$ are (close to) zero?

The block computes the identity, and gradients flow straight back through the shortcut1
The block outputs zero, so the signal dies out in deep networks2
The block can no longer be trained, because all of its gradients vanish3
The block turns into a plain two-layer network without a shortcut4

Timed test