ml-recall

#causal-mask

6 questions from Attention & Transformers, Language Modeling, PyTorch

Timed test

6 questions · 6 min · score and explanations at the end

Take a test

Q 1 / 6Score 0 / 0

Shuffle questions

Question 1: Short answer

easyAI-checkedAttention & Transformers

A GPT-style decoder applies left-to-right (causal) masking to a sequence of 6 tokens: token $t$ may attend only to tokens $1, \dots, t$ . How many of the 36 entries of the $6 \times 6$ attention-weight matrix can be non-zero?

Enter: check / next