#linear-probing

Question 1: Single choice

A model with good pretrained features is adapted to a task whose test data come from a shifted distribution. Kumar et al. compare full fine-tuning (randomly initialized head, all layers trained) with linear probing (frozen features, only the head trained), and recommend LP-FT: linear probing first, then full fine-tuning. Why does LP-FT help?

Linear probing reaches the better in-distribution accuracy, which fine-tuning then keeps1
LP-FT can use a much larger learning rate, which lets the backbone escape the pretrained solution2
Linear probing shrinks the backbone's weights first, which regularizes the later fine-tuning3
Starting from a trained head, fine-tuning distorts the pretrained features less4

Timed test