#distribution-shift

Question 1: Single choice

You have labeled training data and unlabeled inputs collected after deployment. How can you check whether the input distribution has shifted (covariate shift)?

Train a classifier to separate the two sets and check its accuracy on its own training data1
Train a classifier to tell training from deployment inputs; check if its held-out accuracy beats chance2
Compare the model's prediction accuracy on the training set with its accuracy on the deployment inputs3
Check whether the training loss keeps decreasing after the deployment inputs arrive4

Timed test