#cyclic-learning-rate

Question 1: Single choice

mediumAI-checkedDecision Trees & Ensembles

A snapshot ensemble trains one network with the learning-rate schedule in the figure: four cosine cycles of 50 epochs, each restarted at the maximum rate. At which of the marked points (and the corresponding points of the other cycles) is a snapshot for the ensemble taken?

Learning rate (0 to 0.1) against epoch (0 to 200) in four cosine-shaped cycles of 50 epochs, each falling from 0.1 to almost 0 and then jumping back to 0.1. Point A is at epoch 1, B at epoch 26, C at epoch 50 and D at epoch 51.

At C, the end of a cycle, just before the restart1
At A and D, the start of a cycle, where the rate is at its maximum2
At B, halfway through a cycle, where the rate is $\alpha_0 / 2$ 3
At none of them, since a single snapshot is taken after epoch 2004

#cyclic-learning-rate

Timed test