#domain-adaptation

Question 1: Single choice

A small GPT-2 model pretrained on Python code, given the prompt class SimpleAdder {, quickly falls back into Python syntax. NMA fine-tunes it on JavaScript files with

data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

What training objective does mlm=False select?

Causal language modelling: the labels are the inputs themselves, and each next token is predicted1
Masked language modelling: some tokens are hidden and predicted from both sides2
Sequence classification: each file is labelled "JavaScript" or "Python"3
Instruction tuning on prompt–answer pairs, with the loss only on the answers4

Timed test