Question 1: Short answer
easyAI-checkedOptimization & CalculusPer-example gradients are i.i.d. If you increase the minibatch size from 16 to 256, by what factor does the standard deviation of the minibatch gradient shrink?
6 questions from Optimization & Calculus, Training Deep Networks, PyTorch
6 questions · 6 min · score and explanations at the end
Per-example gradients are i.i.d. If you increase the minibatch size from 16 to 256, by what factor does the standard deviation of the minibatch gradient shrink?