Question 1: Short answer
hardAI-checkedLearning Theoryd2l bounds the test error of a fixed classifier on fresh examples with Hoeffding's inequality, , and gets its "roughly 15,000 examples" for at 95% confidence by setting the right-hand side equal to . Using the same recipe, how many test examples are needed to estimate the error to within with probability 99.9%?