Skip to content
ml-recall

#vocabulary-size

4 questions from NLP Fundamentals, Tokenization

Timed test

4 questions · 4 min · score and explanations at the end

Take a test
Q 1 / 4Score 0 / 0

Question 1: Short answer

mediumAI-checkedNLP Fundamentals

Heaps' (Herdan's) law models the vocabulary size as ∣V∣=kNβ|V| = kN^{\beta} for a text of NN tokens. A corpus of 1 million tokens contains 40,000 word types. If β=0.5\beta = 0.5, how many word types do you expect in 100 million tokens of similar text?

a whole number, no separators