#l1-penalty

A logistic regression $P(y = 1 \mid x) = g(w_1 x_1 + w_2 x_2)$ (no bias) is trained on data that the sign of $x_2$ alone classifies perfectly, although the unregularized solution uses both weights. You maximize $\sum_i \log p(y_i \mid x_i) - \frac{C}{2}(|w_1| + |w_2|)$ and increase $C$ from 0. What do you expect to see?