ml-recall

#ppo

2 questions from Policy Gradient & Actor-Critic

Timed test

2 questions · 2 min · score and explanations at the end

Take a test

Q 1 / 2Score 0 / 0

Shuffle questions

Question 1: Short answer

hardAI-checkedPolicy Gradient & Actor-Critic

PPO's clipped objective for one sample is $L = \min\big(r A,\ \operatorname{clip}(r, 1 - \epsilon, 1 + \epsilon)\, A\big)$ with $r = \pi_\theta(a \mid s) / \pi_{\theta_{\text{old}}}(a \mid s)$ . With $\epsilon = 0.2$ and an advantage $A = -0.5$ , the ratio has grown to $r = 1.4$ during the update epochs. What is $L$ ?

Enter: check / next