#small-data

Question 1: Single choice

You receive one tenth of a dataset. Because it is small, you split it into 96% training and 4% validation data, tune many hyperparameter settings on that single validation split, and report a validation accuracy of 0.97. Would the tuned model classify the rest of the data with similar accuracy?

Probably not, since the tiny validation set was reused many times and 0.97 is likely optimistic1
Probably yes, since 0.97 was measured on data the model was not trained on2
Probably yes, since a 96% training split makes overfitting impossible3
Probably not, since a model tuned on a tenth of the data can never generalize4

Timed test