#transformer-decoder

Question 1: Single choice

You want a Transformer for plain language modeling: predict each next token of a text from the tokens before it. Which part of the architecture do you use?

The decoder alone, without cross-attention, with a causal mask on its self-attention1
The encoder alone without masks, so every token can use the whole text2
The full encoder–decoder, with the same text fed to both sides3
The encoder alone, trained to predict tokens that were masked at random positions in the text4

#transformer-decoder

Timed test