#bias-gradient

A dense layer computes $Z = AW + b$ for a batch of 3 examples and 2 units, with the bias row $b$ broadcast to every row. Backpropagation gives the upstream gradient (the $1/B$ of the mean loss is already included):

$D = \frac{\partial L}{\partial Z} = \begin{bmatrix} 0.2 & -0.1 \\ 0.4 & 0.3 \\ -0.1 & 0.5 \end{bmatrix}.$

The first component of $\partial L/\partial b$ is and the second is .