#kernel-ridge-regression

Question 1: Single choice

mediumAI-checkedNearest Neighbors & Kernel Methods

Kernel ridge regression predicts $y(x) = \sum_{t=1}^{n} (\hat\alpha_t / \lambda) K(x_t, x)$ with $\hat{\boldsymbol\alpha} = \lambda(\lambda I + K)^{-1}\mathbf{y}$ , where $K$ is the Gram matrix of the $n$ training inputs. The kernel is $K(x, x') = \exp(-\beta\lVert x - x'\rVert^2 / 2)$ and the inputs are distinct. What happens to the training error as $\lambda \to 0$ ?

It becomes exactly zero, since the fit interpolates the targets1
It tends to the noise variance of the targets2
It is undefined, because the Gram matrix becomes singular3
It tends to the training error of ordinary least squares on the raw inputs4

#kernel-ridge-regression

Timed test