#off-policy

Question 1: Select all that apply

Tabular Q-learning explores with an ε-greedy policy; all Q-values start at 0, and argmax breaks ties by taking the first action. Select all statements that are true.

With ε = 0, the agent keeps taking the tie-break action and, on a sparse-reward map, may never reach the goal or learn anything1
With ε = 1, the agent acts at random, yet the learned Q can still approach $Q^*$ , since the target uses $\max_{a'} Q(s', a')$ 2
With ε = 1, the greedy policy read off the learned Q-table is just as random as the behavior3
A larger ε always earns a higher return during training, because the agent explores more4
ε matters only in the first episode; afterwards the agent acts greedily whatever its value5

Timed test