#num-workers

Loading and augmenting one batch takes exactly 30 ms of CPU time, and one training step on the GPU takes exactly 10 ms. The loop calls loss.item() every step, so Python waits for each step to finish before it asks the DataLoader for the next batch. Ignore all other costs, and assume there are enough CPU cores for all workers.

With num_workers=0, one iteration takes ms. The smallest num_workers for which, once training is under way, an iteration takes only the 10 ms of the GPU step (the GPU never waits for data) is .