#sft

Question 1: Select all that apply

You fine-tune an LLM into a chat assistant. Which statements about learning rates, batch sizes and training data are supported by published results? Select all that apply.

In LIMA's ablations, 16 times more examples of the same quality kept improving the answers1
LoRA tolerates large batch sizes better than full fine-tuning, so batches can grow freely2
LoRA usually needs a higher learning rate than full fine-tuning of the same model3
LIMA made a 65B model a strong assistant with only 1,000 carefully curated examples4

Timed test