#discount-factor

Question 1: Single choice

In FrozenLake with deterministic moves, the agent gets reward 1 only on the move that enters the goal and 0 otherwise, and holes and the goal end the episode. What does value iteration return with discount factor $\gamma = 0$ ?

Value 1 only for states one move from the goal; elsewhere the policy is an arbitrary tie-break1
The same optimal policy as with $\gamma = 0.95$ , only with smaller values everywhere2
Value 1 for every state from which the goal can be reached3
Nothing useful, because value iteration does not converge without discounting4

Timed test