#memory-cell

Question 1: Single choice

mediumAI-checkedRecurrent Neural Networks

In an LSTM the input node $\tilde{\mathbf{C}}_t = \tanh(\cdot)$ already lies in $(-1, 1)$ . Why is $\tanh$ applied again in $\mathbf{H}_t = \mathbf{O}_t \odot \tanh(\mathbf{C}_t)$ ?

The cell state $\mathbf{C}_t$ is a running sum over time steps and can leave $(-1, 1)$ 1
The output gate $\mathbf{O}_t$ can be negative, and the second $\tanh$ flips the sign back2
Without the second $\tanh$ , the gradient through the memory cell would always be zero3
It turns $\mathbf{H}_t$ into a probability distribution over the hidden units4

Timed test