#log-std

Question 1: Single choice

mediumAI-checkedPolicy Gradient & Actor-Critic

A PPO policy for continuous actions is a diagonal Gaussian whose mean comes from a neural network. Many implementations keep a separate parameter vector $\log\sigma$ (not depending on the state) and use $\sigma = \exp(\log\sigma)$ . Why learn $\log\sigma$ rather than $\sigma$ directly?

$\log\sigma$ can take any real value, while $\sigma$ must stay positive1
It makes $\sigma$ depend on the state through the mean network2
The log makes the policy deterministic, which PPO requires3
Sampling actions is only possible when the standard deviation is in log space4

Timed test