#trajectory-probability

Question 1: Single choice

mediumAI-checkedPolicy Gradient & Actor-Critic

The gradient of the log-probability of a trajectory $\tau = (s_0, a_0, \dots, s_{T+1})$ under policy $\pi_\theta$ is used to derive the policy gradient. Why can the policy gradient be estimated without a model of the environment's dynamics?

The log-derivative trick converts the transition probabilities into rewards along the way1
The derivation assumes deterministic transitions, so all their probabilities are equal to 12
Only the policy's log-probabilities depend on $\theta$ ; the environment terms have zero gradient3
The transition probabilities are learned together with the policy and then cancel4

#trajectory-probability

Timed test