Bigram

See also: Machine learning terms

Bigram in Machine Learning

A bigram is a fundamental concept in the field of natural language processing (NLP), a subfield of machine learning. Bigrams are pairs of consecutive words in a given text or sequence of words. They play a vital role in various NLP tasks, such as language modeling, text classification, and sentiment analysis, by capturing the contextual information of words in a language.

Definition and Notation

Formally, a bigram is a tuple of two consecutive words (w1, w2) in a text, where w1 and w2 are words from a given vocabulary. Bigrams can be represented as a matrix, where the rows and columns correspond to the words in the vocabulary, and the matrix cell at position (i, j) contains the frequency or probability of the corresponding word pair. This matrix is often referred to as a bigram probability matrix.

In mathematical notation, a bigram probability can be expressed as P(w2|w1), which denotes the probability of observing word w2 after word w1 in a given text. The probabilities can be estimated using Maximum Likelihood Estimation (MLE) or other smoothing techniques, such as Laplace smoothing or Kneser-Ney smoothing.

Applications of Bigrams in NLP

Bigrams are commonly used in a variety of NLP tasks to capture the contextual relationships between words. Some notable applications include:

• Language Modeling: Bigrams are employed in n-gram language models to predict the probability of a word occurring in a given context. These models can be used for tasks such as speech recognition, machine translation, and text generation.
• Text Classification: Bigrams can be utilized as features in text classification tasks to better capture the word dependencies in the text. For example, in sentiment analysis, the presence of specific bigrams can be indicative of positive or negative sentiment.
• Spell Checking and Correction: Bigram probabilities can be used to assess the likelihood of a given word sequence in the context of a language model. This information can be valuable for identifying and correcting spelling errors or typos.

Explain Like I'm 5 (ELI5)

Imagine you have a bunch of toy blocks, each with a different word on it. A bigram is just two of these blocks placed side by side, making a pair of words. In the world of computers and language, bigrams help us understand how often two words are found together. This information is useful for tasks like guessing what word comes next, figuring out if a sentence is happy or sad, or fixing spelling mistakes.