#mixed-precision

Question 1: Single choice

Mixed-precision training often uses bfloat16 rather than float16, although bfloat16 has fewer mantissa bits. Why is bfloat16 safer for operations such as the $\exp$ inside a softmax?

It has more mantissa bits than float16, so large exponentials are rounded more precisely1
It saturates at its largest finite value instead of overflowing to infinity2
It stores the logarithm of each number, so softmax never needs to exponentiate3
It keeps float32's 8 exponent bits, so $\exp$ overflows only above about 88.7, not 11.14

Timed test