#training-speed

Question 1: Single choice

You train a model whose output layer predicts the next word over a 50,000-word vocabulary, and computing the full softmax at every training step is too slow. What does sampled softmax do?

It estimates the loss from the target word's logit and those of a random sample of other words1
It computes the exact softmax but samples the training examples it is applied to2
It samples the next word from the predicted distribution during generation instead of taking the argmax3
It keeps only the 50 most frequent words in the output layer for good4

Timed test