#log-likelihood-gradient

In one state, a policy is a soft-max over action preferences, one preference per action: $h = (h_L, h_M, h_R) = (1.0, 2.0, 0.0)$ and $\pi(a) = e^{h_a} / \sum_b e^{h_b}$. At time $t = 0$ the agent takes action L, and the episode's return is $G_0 = 2$. REINFORCE updates $h \leftarrow h + \alpha\, G_0 \nabla_h \ln \pi(L)$ with $\alpha = 0.1$. What is the new value of $h_M$, the preference of an action that was not taken?