#square-loss

A linear predictor $W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ is fit with the square loss $L(W) = \frac{1}{2n}\sum_{i=1}^{n}\frac{1}{d_{\text{out}}}\lVert y^{(i)} - W x^{(i)} \rVert_2^2,$ where every input is normalized to $\lVert x^{(i)} \rVert_2 = \sqrt{d_{\text{in}}}$. The bound $L(W + \Delta W) \le L(W) + \operatorname{tr}(G^\top \Delta W) + \tfrac{\lambda}{2}\lVert \Delta W \rVert_{2 \to 2}^2$ holds for every $\Delta W$, with $G$ the gradient and $\lVert\cdot\rVert_{2 \to 2}$ the spectral norm (the largest singular value). For $d_{\text{in}} = 512$ and $d_{\text{out}} = 128$, what is the smallest $\lambda$ this argument gives?