#nadaraya-watson

Question 1: Single choice

mediumAI-checkedNearest Neighbors & Kernel Methods

Binary labels $y_i \in \{-1, +1\}$ . Parzen windows estimate $\hat{p}(\mathbf{x}, y) = \frac{1}{n}\sum_{i:\,y_i = y} k(\mathbf{x}, \mathbf{x}_i)$ , and Nadaraya–Watson gives $f(\mathbf{x}) = \sum_i y_i\, k(\mathbf{x}, \mathbf{x}_i) / \sum_j k(\mathbf{x}, \mathbf{x}_j)$ ; the Parzen density estimate is $\hat{p}(\mathbf{x}) = \frac{1}{n}\sum_j k(\mathbf{x}, \mathbf{x}_j)$ . How are $\hat{p}(\mathbf{x}, +1) - \hat{p}(\mathbf{x}, -1)$ and $f(\mathbf{x})$ related?

The difference is $f(\mathbf{x})$ times $\hat{p}(\mathbf{x}) > 0$ , so the predicted class is always the same1
They are equal for every $\mathbf{x}$ , because both are normalized by the number of training points $n$ 2
They predict the same class only when both classes have the same number of training points3
They predict the same class only for the Gaussian kernel, whose weights integrate to 14

Timed test