#model-based-rl

Question 1: Single choice

An RL agent can either learn a model of the environment (which predicts next states and rewards) and plan with it, or learn a policy or value function directly. What is the main risk of the model-based route when no ground-truth model is available?

It cannot plan ahead, because planning needs the true transition probabilities1
The agent exploits the model's errors, doing well in the model but badly in reality2
It always needs many more environment samples than any model-free method would3
A learned model makes the policy deterministic and stops exploration4

Timed test