#causal-attention

Question 1: Single choice

mediumAI-checkedLLM Fundamentals & Prompting

In machine translation the whole source sentence is known before decoding starts. What is a limitation of a decoder-only Transformer that reads "source + target" as one sequence under a causal mask?

Source tokens attend only to earlier tokens, so the source never gets a bidirectional encoding1
It cannot generate a translation longer than the source sentence2
It cannot condition on the source at all, because a decoder-only model has no cross-attention layer3
It must be trained with a different loss than an encoder–decoder model4

Timed test