Skip to content
ml-recall

#bpe

10 questions from Tokenization

Timed test

10 questions · 10 min · score and explanations at the end

Take a test
Q 1 / 10Score 0 / 0

Question 1: Short answer

easyAI-checkedTokenization

A byte-level BPE tokenizer starts from the 256 possible byte values as its symbols and stops when the vocabulary holds 32,000 symbols. Each merge adds the merged pair as one new symbol and removes nothing. How many merges does it perform?

a whole number