#marginal-likelihood

Question 1: Single choice

mediumAI-checkedProbabilistic & Graphical Models

You learn $\ell$ and $\sigma^2$ of a GP regression model by maximizing the log marginal likelihood twice: once starting from a large $\ell$ with a large $\sigma^2$ , once from a small $\ell$ with a small $\sigma^2$ . The two runs end at different hyperparameters. What is the best explanation?

A slow function with much noise and a wiggly one with little noise both fit, giving separate local optima1
The optimizer must be buggy, since the log marginal likelihood is a convex function of $(\ell, \sigma^2)$ with one optimum2
The complexity term always prefers the smallest $\ell$ , so the result only shows how long each run lasted3
Marginal likelihood can learn the noise variance but not the length-scale, so $\ell$ stays near its start4

#marginal-likelihood

Timed test