#on-policy

Question 1: Single choice

Compared with policy-optimization methods such as PPO, what is the main strength and the main weakness of Q-learning methods such as DQN?

Strength: they can reuse old data off-policy, so they are more sample efficient; weakness: training is less stable1
Strength: they handle continuous actions directly; weakness: they cannot use a replay buffer2
Strength: they need no exploration at all; weakness: they need a model of the environment3
Strength: they directly optimize the expected return, so they are more stable; weakness: they need fresh on-policy data4