#gridworld

A grid world has 6 free cells in 2 rows of 3, surrounded by walls; the goal g is the top-left cell:

g . .
. . .

Actions move up, down, left or right; moving into a wall leaves the agent where it is. Every action whose next cell is the goal gives reward 1 (including bumping into a wall while standing on the goal); all other actions give 0, and the goal is not terminal. Value iteration with $\gamma = 0.99$ converges to the optimal values $V^*$. What is $V^*$ of the bottom-right cell?