Category

Natural Language Processing

226 articlesRSS

Showing 181-226 of 226 articles

Temperature (artificial intelligence)

Temperature is a hyperparameter that controls the randomness of a large language model's output by scaling the model's raw scores, called logits, before they...

Large Language ModelsMachine Learning

Text Classification Models

Text classification models are machine learning systems that assign one or more predefined categorical labels to a span of natural language text, such as...

AI Models

Text Generation Models

Text generation models are language models trained to produce coherent natural-language text by predicting tokens one at a time, each conditioned on the...

AI Models

Text summarization

Text summarization is the natural language processing (NLP) task of automatically producing a shorter version of one or more documents that preserves the most...

Deep LearningMachine Learning

Text-to-Speech

Text-to-speech (TTS) refers to artificial intelligence systems that convert written text into natural-sounding spoken audio. Modern TTS systems use deep...

Artificial Intelligence

Text2Text Generation Models

Text-to-text (text2text) generation models are a family of neural network systems that frame many natural language processing tasks as a single problem: given...

AI Models

The Pile (dataset)

The Pile is an 825.18 GiB (approximately 886 GB) English text corpus designed for training large language models, assembled from 22 diverse, high-quality...

Data & DatasetsMachine Learning

Token

A token is the basic unit of text that a language model reads and writes: a word, a subword fragment, a single character, or a byte, produced by splitting text...

Deep LearningMachine Learning

Token Classification Models

Token classification models are natural language processing systems that assign a discrete label to every token in an input sequence, where a token is...

AI Models

Tokenization

Tokenization is the process of breaking text into smaller units called tokens, which serve as the fundamental input to natural language processing (NLP)...

Large Language ModelsMachine Learning

Tool use

Tool use is the ability of large language models (LLMs) and AI systems to interact with external tools, APIs, databases, and software functions during...

AI AgentsArtificial Intelligence

Top-p and top-k sampling

Top-p sampling, also called nucleus sampling, is a text-generation decoding method for large language models (LLMs) that, at each step, samples the next token...

Large Language ModelsMachine Learning

Top-p sampling

Top-p sampling, also called nucleus sampling, is a stochastic decoding method for text generation in which the model samples from the smallest possible set of...

Large Language ModelsMachine Learning

Topic model

A topic model is a statistical model that discovers the abstract "topics" hidden in a collection of documents, where each document is represented as a mixture...

Machine LearningStatistics

ToxiGen

ToxiGen is a large-scale, machine-generated dataset designed for adversarial and implicit hate speech detection. Created by Thomas Hartvigsen, Saadia Gabriel,...

AI BenchmarksAI Ethics

Translation Models

See also: Natural Language Processing Models and Tasks Translation models are computational systems that convert text or speech from a source language into a...

AI Models

Trigram

See also: N-gram, Bigram, Language model A trigram is a contiguous sequence of three items (most often three words) drawn from a sample of text or speech, and...

Machine Learning

TriviaQA

TriviaQA is a large-scale reading comprehension and question answering dataset of over 650,000 question-answer-evidence triples, introduced in 2017 by Mandar...

AI Benchmarks

TruthfulQA

TruthfulQA is a benchmark designed to measure whether large language models (LLMs) generate truthful answers to questions. Created by Stephanie Lin, Jacob...

AI BenchmarksAI Safety

Unidirectional language model

See also: Machine learning terms A unidirectional language model is a language model that predicts each token using only the tokens that come before it in the...

Large Language Models

Universal Speech Model

The Universal Speech Model (USM) is a family of large multilingual speech models developed by Google Research that performs automatic speech recognition (ASR)...

AI ModelsSpeech & Audio AI

Vector embeddings

See also: AI terms Vector embeddings are dense numerical representations of objects (text, images, audio, video, code, graphs, or any structured data) that map...

Information Retrieval

Voyage AI

Voyage AI is an artificial intelligence company that builds state-of-the-art embedding and reranking models for retrieval-augmented generation and semantic...

AI CompaniesInformation Retrieval

Voyage-3

Voyage-3 is a family of general-purpose text embedding models developed by Voyage AI, launched in September 2024 with voyage-3 and voyage-3-lite [1], expanded...

AI ModelsAnthropic

Wav2Vec

Wav2Vec is a family of self-supervised learning models from Meta AI (formerly Facebook AI Research) that learn speech representations directly from raw audio...

Deep LearningMeta AI

Whisper

Whisper is an open-source family of automatic speech recognition (ASR) models developed by OpenAI and first released on September 21, 2022.[^1] Built on an...

Deep LearningOpenAI

WinoGrande

WinoGrande is a large-scale benchmark for commonsense reasoning consisting of 44,000 binary fill-in-the-blank pronoun resolution problems, built to test...

AI Benchmarks

Winograd Schema Challenge

The Winograd Schema Challenge (WSC) is a commonsense reasoning test in which a system must resolve an ambiguous pronoun in a short sentence where the correct...

AI Benchmarks

Word Embedding

A word embedding is a learned representation of text in which words are mapped to dense vectors of real numbers in a continuous vector space, so that words...

Deep LearningMachine Learning

Word error rate

Word error rate (WER) is the standard metric for measuring the accuracy of an automatic speech recognition (ASR) system, expressed as the fraction of words the...

Machine LearningModel Evaluation

WordPiece

WordPiece is a subword tokenization algorithm that builds a fixed-size vocabulary of word pieces by repeatedly merging the symbol pair whose combination most...

Large Language Models

Wordtune

AI writing and reading assistant Developer October 27, 2020 Platform Proprietary (NLP models) Headquarters 10 million+ (2025) Website Wordtune is an...

AI Tools & Products

WritingBench

WritingBench is a comprehensive benchmark for evaluating the generative writing capabilities of large language models (LLMs) across diverse real-world writing...

AI BenchmarksLarge Language Models

XLM-RoBERTa

XLM-RoBERTa (often abbreviated XLM-R) is a multilingual masked language model developed by Facebook AI Research (now Meta AI) and released in November 2019. It...

Large Language Models

XLNet

XLNet is a generalized autoregressive pretraining method for natural language processing that combines the strengths of autoregressive and autoencoding...

Deep LearningMachine Learning

Z.ai

Public company (HKEX: 2513) Industry June 11, 2019 Founders Haidian District, Beijing, China Key people GLM (General Language Model) family, ChatGLM,...

AI CompaniesChinese AI

ZebraLogic

ZebraLogic is a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs). Developed by researchers at the Allen Institute...

AI BenchmarksLarge Language Models

Zero-Shot Classification Models

Zero-shot classification models are machine learning systems that assign input text to a set of candidate categories without having seen labeled training...

AI Models

chrF

chrF is a machine translation evaluation metric that scores a candidate translation by counting the character n-grams it shares with one or more reference...

Machine LearningModel Evaluation

fastText

fastText is an open-source library for learning word embeddings and performing text classification, developed by Facebook AI Research (FAIR) and released to...

Machine LearningOpen Source AI

mT5

mT5 (multilingual T5) is a transformer-based encoder-decoder language model released by Google Research in October 2020 that covers 101 languages in a single...

Large Language ModelsOpen Source AI

olmOCR

olmOCR is an open toolkit and vision-language model from the Allen Institute for AI (Ai2) that turns PDFs and document images into clean, structured plain text...

Computer VisionOpen Source AI

text-embedding-3

text-embedding-3 is a family of third-generation text embedding models that OpenAI released on January 25, 2024. It consists of two models,...

Information RetrievalOpenAI

tiktoken

tiktoken is an open source byte pair encoding (BPE) tokenizer library released by OpenAI in December 2022 that converts text into the integer token sequences...

Open Source AIOpenAI

word2vec

word2vec is a family of shallow neural network models, published by Tomas Mikolov and colleagues at Google in 2013, that learns dense vector representations of...

Machine Learning

Łukasz Kaiser

Łukasz Kaiser is a Polish computer scientist and researcher at OpenAI who is one of the eight co-authors of the 2017 paper "Attention Is All You Need," the...

People