Perplexity: Difference between revisions

{{see also|language models}}

Perplexity is an important measurement for determining how good a [[language model]] is. Essentially, it quantifies the quality of the model's predictions by evaluating the inverse probability of the [[test set]], normalized by the number of words, or by calculating the average number of bits required to encode a single word through [[cross-entropy]]. Perplexity can be perceived as the [[weighted branching factor]], and a high perplexity score represents a higher degree of confusion in the model's next-word predictions, while a low perplexity score implies greater confidence in the model's output.