Skip to content
ml-recall

#numerical-stability

11 questions from Information Theory, Linear Models & Regularization, Logistic Regression & Softmax, Neural Network Basics, Training Deep Networks, Attention & Transformers, PyTorch

Timed test

11 questions · 11 min · score and explanations at the end

Take a test
Q 1 / 11Score 0 / 0

Question 1: Short answer

mediumAI-checkedInformation Theory

This is the d2l entropy function. What does the last line return?

import torch

def nansum(x):
    return x[~torch.isnan(x)].sum()

def entropy(p):
    return nansum(-p * torch.log2(p))

entropy(torch.tensor([0.5, 0.25, 0.25, 0.0]))

a number (bits)