Skip to content
ml-recall

#momentum

12 questions from Optimization & Calculus, Training Deep Networks

Timed test

12 questions · 12 min · score and explanations at the end

Take a test
Q 1 / 12Score 0 / 0

Question 1: Short answer

easyAI-checkedOptimization & Calculus

Plain SGD works well with η=0.1\eta = 0.1. You switch to momentum, vt=βvt−1+gt\mathbf{v}_t = \beta\mathbf{v}_{t-1} + \mathbf{g}_t, xt=xt−1−ηvt\mathbf{x}_t = \mathbf{x}_{t-1} - \eta\mathbf{v}_t, with β=0.9\beta = 0.9. Where the gradient is roughly constant, which η\eta gives the same step size as before?

a number