#safety-evaluation

Question 1: Single choice

Perez et al. (2022) red-team a chatbot with a second "red" language model that writes test questions, while a classifier flags offensive replies. When the red LM is trained with RL to elicit offensive replies, a KL penalty with weight $\alpha$ keeps it close to its initial distribution. What happened as $\alpha$ was lowered?

Fewer of its questions drew offensive replies, but the questions became more diverse1
Its questions became more diverse and also drew offensive replies more often2
Nothing changed, because the KL penalty affects only the training speed3
More of its questions drew offensive replies, but the questions became less diverse4

Timed test