#truncated-bptt

Question 1: Select all that apply

mediumAI-checkedRecurrent Neural Networks

Besides gradient clipping, which of these help against exploding gradients when an RNN is trained with backpropagation through time? Select all that apply.

Truncating backpropagation after a fixed number of time steps (detaching the hidden state)1
Lowering the learning rate, which shrinks every update $\eta\mathbf{g}$ 2
Backpropagating through the full sequence instead of truncating it3
Initializing $\mathbf{W}_\textrm{hh}$ with eigenvalues well above 1, so the signal does not fade4

Timed test