Skip to content
ml-recall

#relative-position

1 question from Attention & Transformers

Timed test

1 question · 1 min · score and explanations at the end

Take a test
Q 1 / 1Score 0 / 0

Question 1: Short answer

mediumAI-checkedAttention & Transformers

Relative position representations (Shaw et al., 2018) learn one embedding per query–key offset j−ij - i, with the offset clipped to [−k,k][-k, k]. With clipping distance k=4k = 4, how many distinct offset embeddings does a layer learn for its keys, whatever the sequence length?

a whole number