#large-vocabulary

Question 1: Select all that apply

mediumAI-checkedLogistic Regression & Softmax

You use softmax regression to predict the next word from a 512-dimensional feature vector, over a vocabulary of 50,000 words. Which problems does the large vocabulary cause? Select all that apply.

The softmax outputs no longer sum to 1 when there are very many classes1
The cross-entropy loss stops being convex in the weights2
The output layer alone has about 25.6 million weights3
Every probability or loss evaluation needs all 50,000 logits for the softmax denominator4
Rare words occur in few training examples, so their weights are poorly estimated5

Timed test