Skip to content
ml-recall

#out-of-vocabulary

3 questions from Tokenization

Timed test

3 questions · 3 min · score and explanations at the end

Take a test
Q 1 / 3Score 0 / 0

Question 1: True or false

easyAI-checkedTokenization

A word-level vocabulary of the 3,000 most frequent words maps the unseen word "wampinuk" to UNK. True or false: a BPE tokenizer whose base vocabulary contains every character of the training corpus must also map "wampinuk" to a single UNK token.

Choose one option