#mode-seeking

Question 1: Single choice

Maximising the ELBO over a variational density $q$ minimises $\mathrm{KL}(q \,\|\, p) = \mathbb{E}_q[\log q(x) - \log p(x)]$ to the true posterior $p$ ; this is also what trains the encoder of a VAE. Suppose $p$ is bimodal with two well-separated modes, but $q$ can only be unimodal (e.g. Gaussian). What does the optimal $q$ typically look like?

It sits on one of the two modes and largely ignores the other1
It spreads over both modes, with its peak in the low-density gap between them2
It matches the overall mean and variance of $p$ 3
It equals $p$ , since the KL divergence is minimal at zero4

Timed test