#elbo

Question 1: Single choice

hardAI-checkedProbabilistic & Graphical Models

Mean-field variational inference approximates $p(\mathbf{y} \mid \mathbf{x})$ by $q(\mathbf{y} \mid \mathbf{x}) = \prod_{i=1}^d q_i(y_i \mid \mathbf{x})$ and maximises the ELBO $\mathbb{E}_q[\log p(\mathbf{x}, \mathbf{y}) - \log q(\mathbf{y} \mid \mathbf{x})]$ one factor at a time. With $q_2, \ldots, q_d$ held fixed, what is the optimal $q_1(y_1 \mid \mathbf{x})$ ?

$q_1(y_1 \mid \mathbf{x}) = p(y_1 \mid \mathbf{x})$ , the exact posterior marginal1
$q_1(y_1 \mid \mathbf{x}) \propto \exp(\mathbb{E}_{q_2 \cdots q_d}[\log p(\mathbf{x}, \mathbf{y})])$ 2
$q_1(y_1 \mid \mathbf{x}) \propto \mathbb{E}_{q_2 \cdots q_d}\left[p(\mathbf{x}, \mathbf{y})\right]$ 3
$q_1(y_1 \mid \mathbf{x}) \propto p(\mathbf{x}, y_1, \bar y_2, \ldots, \bar y_d)$ with $\bar y_i$ the means under $q_i$ 4

Timed test