Skip to content
ml-recall

#gpt

2 questions from Attention & Transformers, Language Modeling

Timed test

2 questions · 2 min · score and explanations at the end

Take a test
Q 1 / 2Score 0 / 0

Question 1: Short answer

easyAI-checkedAttention & Transformers

A GPT-style decoder applies left-to-right (causal) masking to a sequence of 6 tokens: token tt may attend only to tokens 1,…,t1, \dots, t. How many of the 36 entries of the 6×66 \times 6 attention-weight matrix can be non-zero?

a whole number