#pretraining-objective

Question 1: Single choice

For pretraining, a DNA language model uses DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False). What training objective does mlm=False select?

Sequence classification — predict one label per sequence1
No objective — the collator only pads and skips the loss2
Causal language modelling — predict each token from the tokens before it3
Masked language modelling — predict about 15% of the tokens, hidden at random4

#pretraining-objective

Timed test