Skip to content
ml-recall

#gpt-2

5 questions from Attention & Transformers, Tokenization, LLM Fundamentals & Prompting

Timed test

5 questions · 5 min · score and explanations at the end

Take a test
Q 1 / 5Score 0 / 0

Question 1: Short answer

mediumAI-checkedAttention & Transformers

The feed-forward module of each GPT-2 small block (embedding size 768). What does the last line print?

emb_dim = 768
ff = nn.Sequential(
    nn.Linear(emb_dim, 4 * emb_dim),
    nn.GELU(approximate="tanh"),
    nn.Linear(4 * emb_dim, emb_dim),
)
print(sum(p.numel() for p in ff.parameters()))

an integer