#hidden-units

Question 1: Single choice

An MLP for 10-class Fashion-MNIST has a hidden layer with a single ReLU unit, $784 \to 1 \to 10$ . Why is this a bad design?

The network can predict at most two of the ten classes, one on each side of the ReLU's kink1
A single hidden unit makes the network far too large to train in a reasonable amount of time2
Softmax needs at least as many hidden units as there are classes to produce normalized probabilities3
All logits depend on one number, and if that ReLU unit dies, no gradient reaches the first layer4