#overcomplete

Question 1: Select all that apply

An autoencoder with input size $d$ and code size $d'$ is trained only to minimise reconstruction error. With $d' \geq d$ and no other constraint it can reach zero error by learning the identity, which teaches it nothing useful. Which changes rule out this trivial solution? Select all that apply.

Make the code undercomplete, $d' < d$ , so the input has to pass through a bottleneck1
Replace the squared error with a cross-entropy reconstruction loss on the same clean target2
Feed a corrupted copy $\tilde{x}$ of the input and score the output against the clean $x$ 3
Add more units to the code layer, so the network has more capacity to learn useful features4
Keep $d' \geq d$ but penalise the code so that only a few units are active for each input5

Timed test