#learning-rate-schedule

Question 1: Single choice

For a convex objective, stochastic gradients bounded by $L$ and a start at distance $r$ from the optimum, SGD with step sizes $\eta_t$ satisfies $E[R(\bar{\mathbf{x}})] - R^* \le \frac{r^2 + L^2 \sum_{t=1}^T \eta_t^2}{2\sum_{t=1}^T \eta_t}.$ Which schedule makes this bound go to 0 as $T \to \infty$ ?

$\eta_t = \eta_0 / \sqrt{t}$ 1
$\eta_t = \eta_0$ , a constant2
$\eta_t = \eta_0 e^{-\lambda t}$ 3
$\eta_t = \eta_0 / t^2$ 4

#learning-rate-schedule

Timed test