#code-generation

Question 1: Single choice

You fine-tune a code model and evaluate it with token-level accuracy: the fraction of generated tokens that exactly match a reference solution. Why is this a poor metric for code generation?

Many different programs do the same thing, so correct code can score low and similar-looking broken code high1
Accuracy can only be computed for classifiers, not for models that generate sequences2
Accuracy is too slow to compute over long source files during evaluation3
Accuracy rewards longer outputs, so the model learns to pad its code4

Timed test