#weight-shapes

Question 1: Select all that apply

mediumAI-checkedRecurrent Neural Networks

You train a vanilla RNN with one hidden layer, $h_t = \tanh(W x_t + U h_{t-1} + b)$ , and an output layer $y_t = V h_t + c$ . The input vectors have size 200, the output layer has size 200 and the hidden size is 100. Which statements are true? Select all that apply.

The matrix $U$ between the hidden states of consecutive time steps has shape $200 \times 200$ 1
The hidden state $h_t$ is a 100-dimensional vector2
If a bidirectional RNN combines its forward and backward hidden states by element-wise addition, both RNNs need the same hidden size3
The word2vec skip-gram model is likely to suffer from vanishing gradients4
In the forward pass, the hidden state at each step is computed by multiplying the input $x_t$ by $W$ and applying a nonlinearity5

Timed test