#adapters

Question 1: Single choice

Prefix tuning and adapters both fine-tune a frozen transformer with few new parameters. Where does each put its task-specific parameters?

Prefix tuning: key/value vectors prepended at every layer; adapters: small bottleneck layers in each block1
Prefix tuning: small bottleneck layers inside each block; adapters: key/value vectors prepended at every layer2
Both retrain only the final output layer, with different learning rates3
Both write task knowledge into the attention mask instead of adding parameters4