Question 1: Short answer
hardAI-checkedAttention & TransformersViT-Base uses nn.MultiheadAttention(embed_dim=768, num_heads=12) (with biases) in each encoder
layer. How many parameters does this attention module have?
#vit
2 questions from Attention & Transformers
Timed test
2 questions · 2 min · score and explanations at the end
Q 1 / 2Score 0 / 0