Skip to content
ml-recall

#add-one-smoothing

1 question from Language Modeling

Timed test

1 question · 1 min · score and explanations at the end

Take a test
Q 1 / 1Score 0 / 0

Question 1: Short answer

mediumAI-checkedLanguage Modeling

On the corpus

<s> I am Gustave </s>
<s> Gustave I am </s>
<s> I love natural language processing </s>

the word "love" occurs once, always followed by "natural", so the unsmoothed estimate is P(natural∣love)=1P(\text{natural} \mid \text{love}) = 1. With add-one smoothing, P(wi∣wi−1)=count(wi−1 wi)+1count(wi−1)+∣V∣P(w_i \mid w_{i-1}) = \frac{\text{count}(w_{i-1}\, w_i) + 1}{\text{count}(w_{i-1}) + |V|}, and a vocabulary of ∣V∣=8|V| = 8 word types (the 7 words plus </s>), what is P(natural∣love)P(\text{natural} \mid \text{love})?

a probability, 3 decimals