Skip to content
ml-recall

#gelu

1 question from Neural Network Basics

Timed test

1 question · 1 min · score and explanations at the end

Take a test
Q 1 / 1Score 0 / 0

Question 1: Short answer

mediumAI-checkedNeural Network Basics

BERT's feed-forward layers use the GELU activation, GELU(x)=x Φ(x)\mathrm{GELU}(x) = x\,\Phi(x), where Φ\Phi is the standard normal CDF, instead of ReLU. What is GELU(−1)\mathrm{GELU}(-1)? Round to 3 decimals.

a number, 3 decimals (it may be negative)