Question 1: Short answer
easyAI-checkedOptimization & CalculusPer-example gradients are i.i.d. If you increase the minibatch size from 16 to 256, by what factor does the standard deviation of the minibatch gradient shrink?
2 questions from Optimization & Calculus
2 questions · 2 min · score and explanations at the end
Per-example gradients are i.i.d. If you increase the minibatch size from 16 to 256, by what factor does the standard deviation of the minibatch gradient shrink?