Skip to content
ml-recall

#tokens-per-word

1 question from Tokenization

Timed test

1 question · 1 min · score and explanations at the end

Take a test
Q 1 / 1Score 0 / 0

Question 1: Short answer

easyAI-checkedTokenization

A textbook runs a large multilingual BPE tokenizer on a recipe: the 14-word English version becomes 19 tokens, while the 16-word Spanish original becomes 33 tokens. How many tokens per word does the Spanish text need (2 decimals)?

a number with 2 decimals