#model-parallelism

Question 1: Single choice

mediumAI-checkedDeep Learning Frameworks

Your model fits comfortably on one GPU, and you have 8 GPUs to speed up training. Why is data parallelism usually preferred over model parallelism here?

Each GPU runs a full replica on its share of the batch; only gradients are exchanged1
Data parallelism needs no communication at all between the GPUs2
Model parallelism can never use more than two GPUs3
Data parallelism lets you train a model that is larger than the memory of any single GPU4

Timed test