#cross-entropy

Q 1 / 29Score 0 / 0

Shuffle questions

Question 1: Single choice

When training a classifier, the data distribution $P$ is fixed and only the model distribution $Q$ changes. Why does minimizing the cross-entropy $\mathrm{CE}(P, Q)$ give the same $Q$ as minimizing $D_{\mathrm{KL}}(P \| Q)$ ?

$\mathrm{CE}(P, Q) = H(P) + D_{\mathrm{KL}}(P \| Q)$ , and $H(P)$ does not depend on $Q$ 1
$\mathrm{CE}(P, Q) = H(Q) + D_{\mathrm{KL}}(P \| Q)$ , and $H(Q)$ stays constant in training2
The KL divergence is symmetric, so it equals the cross-entropy in either direction3
Both quantities reach their minimum when $Q$ is the uniform distribution4

Keys: 1–9 select · Enter check / next

#cross-entropy

Timed test