#hessian

Question 1: Single choice

$f(x, y) = x^2 - y^2$ has a saddle point at the origin. Gradient descent with a small learning rate $\eta$ starts at $(1, 10^{-6})$ . Why does it not end at the saddle?

Along $y$ the update is $y \leftarrow (1 + 2\eta)y$ , so the tiny offset grows geometrically1
The gradient at the origin is nonzero along $y$ , which pushes the iterate away from it2
It does end there, because every start with $x \neq 0$ is drawn to the critical point3
Escaping needs curvature information, so only Newton-type methods can ever leave a saddle point4

Timed test