Skip to content
ml-recall

#embedding

2 questions from Text Representation & Embeddings, Tokenization

Timed test

2 questions · 2 min · score and explanations at the end

Take a test
Q 1 / 2Score 0 / 0

Question 1: Short answer

easyAI-checkedText Representation & Embeddings

A vocabulary has 4 tokens, and the input layer of a model has the weight matrix

W=[123456789101112]\mathbf{W} = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \\ 10 & 11 & 12 \end{bmatrix}

(one row per token). The token with index 2, counting from 0, is one-hot encoded as e=(0,0,1,0)\mathbf{e} = (0, 0, 1, 0). What is e⊤W\mathbf{e}^\top\mathbf{W}?

three numbers, e.g. (1, 2, 3)