#lower-bound

Question 1: Single choice

A distribution $p(x) = \tilde{p}(x) / Z$ has an intractable normaliser $Z$ . Variational inference picks a tractable $q$ and minimises $J(q) = \sum_x q(x) \log \frac{q(x)}{\tilde{p}(x)}.$ What does $J(q)$ tell you about $\log Z$ ?

$\log Z \ge -J(q)$ , with equality when $q = p$ 1
$\log Z \ge J(q)$ , with equality when $q = p$ 2
$\log Z \le -J(q)$ , so $-J(q)$ is an upper bound3
$\log Z = -J(q)$ for every choice of $q$ 4

Timed test