ml-recall

#distributional-similarity

1 question from Information Theory

Timed test

1 question · 1 min · score and explanations at the end

Take a test

Q 1 / 1Score 0 / 0

Shuffle questions

Question 1: Single choice

mediumAI-checkedInformation Theory

Word distributions estimated from small counts often contain zero probabilities. Which of these measures between two distributions $p$ and $q$ can become infinite because of such a zero?

The KL divergence $D(p \,\|\, q)$ 1
The $L_1$ distance $\sum_i \lvert p_i - q_i \rvert$ 2
The information radius $D(p \,\|\, m) + D(q \,\|\, m)$ with $m = (p + q)/2$ 3
None of them, since probabilities never exceed 14

Keys: 1–9 select · Enter check / next