Skip to content
ml-recall

#layer-normalization

4 questions from Training Deep Networks, Attention & Transformers

Timed test

4 questions · 4 min · score and explanations at the end

Take a test
Q 1 / 4Score 0 / 0

Question 1: Single choice

hardAI-checkedTraining Deep Networks

A hand-written layer normalization puts its smoothing term outside the square root. What does the last line print?

import torch

x = torch.tensor([2.0, 2.0, 2.0], requires_grad=True)   # all features equal
mean = x.mean()
var = x.var(correction=0)
z = (x - mean) / (torch.sqrt(var) + 1e-3)
z.sum().backward()
print(x.grad)
Choose one option