#chrf

Question 1: Single choice

mediumAI-checkedNLP Tasks

Two papers report BLEU on the same test set, 27.1 and 28.4, but they tokenized the outputs differently (for example, one splits "don't" and hyphenated words, the other does not). Why can't the two numbers be compared directly?

BLEU counts word n-gram matches, so its value depends on the tokenization1
BLEU is only defined for a single reference, and the papers may have used several2
BLEU scores above 25 are saturated and no longer distinguish systems3
BLEU measures recall, and recall changes when the candidate length changes4

Timed test