#decoder

Question 1: Single choice

mediumAI-checkedRecurrent Neural Networks

An RNN decoder that copies the encoder's final hidden state as its initial state needs the same number of layers and hidden units as the encoder. Your encoder has 2 layers of 256 units and your decoder 1 layer of 512 units. How can you initialize the decoder's state?

Map the encoder state to the decoder's shape with a learned layer, e.g. $\tanh(\mathbf{W}\mathbf{h})$ 1
Copy the state as it is; the framework broadcasts the 256 hidden units to the 512 the decoder expects2
You cannot; an encoder and a decoder must always have identical layer shapes3
Use the input embedding of the last source token as the decoder's initial state4

Timed test