Question 1: True or false
mediumAI-checkedDeep Learning FrameworksTrue or false: in data-parallel training, synchronizing the gradients across GPUs can only start after backpropagation has computed the gradients of every layer.
3 questions from Deep Learning Frameworks
3 questions · 3 min · score and explanations at the end
True or false: in data-parallel training, synchronizing the gradients across GPUs can only start after backpropagation has computed the gradients of every layer.