Skip to content
ml-recall

#exponential-lr

1 question from Training Deep Networks

Timed test

1 question · 1 min · score and explanations at the end

Take a test
Q 1 / 1Score 0 / 0

Question 1: Short answer

mediumAI-checkedTraining Deep Networks

To find a good learning rate, you train for 500 iterations while growing the learning rate exponentially, record the loss at each iteration, and later look for where the loss shoots up:

opt = torch.optim.SGD(model.parameters(), lr=1e-5)
sched = torch.optim.lr_scheduler.ExponentialLR(opt, gamma=g)
for step in range(500):
    ...                # train on one batch, record the loss
    opt.step()
    sched.step()

Which value of g makes the learning rate reach 10 after the 500 scheduler steps? Give 4 decimals.

A number with 4 decimals, e.g. 1.0123