#computational-graph

Question 1: Single choice

A one-hidden-layer network computes $\mathbf{z} = \mathbf{W}^{(1)}\mathbf{x} + \mathbf{b}^{(1)}$ , $\mathbf{h} = \phi(\mathbf{z})$ and $\mathbf{o} = \mathbf{W}^{(2)}\mathbf{h}$ , with objective $J = l(\mathbf{o}, y) + \frac{\lambda}{2}\left(\lVert\mathbf{W}^{(1)}\rVert_F^2 + \lVert\mathbf{W}^{(2)}\rVert_F^2\right)$ ; the bias is not regularized. What is $\partial J / \partial \mathbf{b}^{(1)}$ ?

$\left({\mathbf{W}^{(2)}}^\top \frac{\partial J}{\partial \mathbf{o}}\right) \odot \phi'(\mathbf{z})$ 1
$\left({\mathbf{W}^{(2)}}^\top \frac{\partial J}{\partial \mathbf{o}}\right) \odot \phi'(\mathbf{z}) + \lambda\mathbf{b}^{(1)}$ 2
$\left(\left({\mathbf{W}^{(2)}}^\top \frac{\partial J}{\partial \mathbf{o}}\right) \odot \phi'(\mathbf{z})\right)\mathbf{x}^\top$ 3
${\mathbf{W}^{(2)}}^\top \frac{\partial J}{\partial \mathbf{o}}$ 4

#computational-graph

Timed test