#vanishing-gradient

Question 1: Single choice

In the network X → A → C (weights $W_{XA} = 1$ , $W_{AC} = 1$ , sigmoid units, no thresholds) with input $X = 60$ and a nonzero output error term $\delta_C \approx 0.053$ , backpropagation gives $\Delta W_{XA} = \delta_A X$ with $\delta_A = o_A(1 - o_A)\,W_{AC}\,\delta_C$ . Why is $\Delta W_{XA}$ practically zero?

Unit A is saturated, so $o_A(1 - o_A) \approx 10^{-26}$ 1
The other hidden unit has weight $W_{BC} = 0$ 2
The output error $d - o_C$ is zero3
The large input $X = 60$ makes the update round off to zero4

Timed test