Skip to content
ml-recall

#probability-ratio

1 question from Policy Gradient & Actor-Critic

Timed test

1 question · 1 min · score and explanations at the end

Take a test
Q 1 / 1Score 0 / 0

Question 1: Short answer

hardAI-checkedPolicy Gradient & Actor-Critic

PPO's clipped objective for one sample is L=min⁡(rA, clip⁡(r,1−ϵ,1+ϵ) A)L = \min\big(r A,\ \operatorname{clip}(r, 1 - \epsilon, 1 + \epsilon)\, A\big) with r=πθ(a∣s)/πθold(a∣s)r = \pi_\theta(a \mid s) / \pi_{\theta_{\text{old}}}(a \mid s). With ϵ=0.2\epsilon = 0.2 and an advantage A=−0.5A = -0.5, the ratio has grown to r=1.4r = 1.4 during the update epochs. What is LL?

a number, 1 decimal