#wordpiece

Question 1: Single choice

Someone proposes skipping trained subword tokenizers: just split text into two-character chunks and one-hot encode them, which also never meets an unknown word. What does a trained subword tokenizer such as BPE or WordPiece offer that this scheme lacks?

Units chosen from corpus frequencies: common words or pieces become single tokens, so sequences are shorter and more meaningful1
Dense vectors that capture meaning, whereas one-hot codes are sparse and carry no similarity2
A guaranteed vocabulary smaller than the 676 possible two-letter combinations3
No need for an embedding layer, because the token ids can be fed to the model directly4

Timed test