#model-selection

Timed test

12 questions · 12 min · score and explanations at the end

Q 1 / 12Score 0 / 0

Shuffle questions

Question 1: Single choice

A team tries 40 hyperparameter settings, keeps the one with the best test-set accuracy (91%), and reports 91% as the accuracy to expect on new data. What is wrong?

Choosing by test accuracy makes 91% optimistic; settings should be compared on a validation set1
Nothing, because choosing among candidate models is the purpose of the test set2
The setting should have been chosen by training accuracy instead, since the training set is the largest3
The only problem is that 40 settings are too few to find a good one4

Keys: 1–9 select · Enter check / next