#checkpointing

Question 1: Single choice

Training a deep network runs out of GPU memory because the activations cached for backpropagation are too large. Which technique trades extra computation for less memory?

Keep only some activations and recompute the others during the backward pass1
Double the batch size so that fewer optimizer steps are needed per epoch2
Run the backward pass before the forward pass so nothing has to be cached3
Raise the learning rate so that training finishes in fewer epochs4

Timed test