#kernel-width

Question 1: Single choice

mediumAI-checkedNearest Neighbors & Kernel Methods

You tune the width σ of a Gaussian kernel in Nadaraya–Watson regression by minimizing the training error $\sum_i (f(\mathbf{x}_i) - y_i)^2$ , where each $f(\mathbf{x}_i)$ is computed with the pair $(\mathbf{x}_i, y_i)$ itself still in the data. Where does σ go?

To 0: each point's own weight dominates, so $f(\mathbf{x}_i) \to y_i$ and the noise is fit1
To ∞: the estimate then becomes the global mean of all labels, which minimizes the squared error2
To the noise level of the labels, which balances bias against variance3
Nowhere in particular, since the training error does not depend on σ4

Timed test