#loss

Question 1: Single choice

mediumAI-checkedRecurrent Neural Networks

In seq2seq training, target sentences are padded with <pad> up to num_steps tokens. What goes wrong if the cross-entropy loss is averaged over all positions, padding included?

The easy <pad> targets count as real tokens, diluting the loss and wasting gradient on padding1
Nothing: <pad> has a fixed index, so its cross-entropy term is always exactly zero2
The encoder can no longer read the source sentence, because the padding hides its final hidden state3
Training stops with a shape error, because padded and unpadded targets differ in length4

Timed test