Machine Learning

Softmax

See also: Machine learning terms The softmax function is a mathematical function that converts a vector of real numbers, often called logits, into a...

Deep LearningNeural Networks

Sparse Coding

Sparse coding is a representation learning principle in which a signal is encoded as a linear combination of a small number of elements drawn from a (usually...

InterpretabilityNeural Networks

Sparse Feature

A sparse feature is a feature in machine learning whose values are predominantly zero or empty across a dataset. Google's Machine Learning Crash Course defines...

Data & Datasets

Sparse Representation

Sparse representation is a data encoding strategy in which most values in a vector, matrix, or tensor are zero (or near zero), with only a small fraction of...

Data & Datasets

Sparse Vector

A sparse vector is a vector in which most of the elements are zero, in contrast to a dense vector, in which most elements are non-zero. Google's Machine...

Computer ScienceMathematics

Sparse attention

Sparse attention is a family of techniques that cut the computational and memory cost of the attention mechanism in transformer models by letting each token...

Deep LearningModel Architecture

Sparse autoencoder

A sparse autoencoder (SAE) is a neural network that adds a sparsity penalty to an autoencoder's training loss so that only a small number of hidden units...

Deep LearningInterpretability

Sparse upcycling

Sparse upcycling is a technique for building a sparsely activated mixture of experts (MoE) model by initializing it from an already trained dense Transformer...

Reinforcement Learning

Sparsity

See also: Machine learning terms Sparsity is the property of a data structure, model, or representation in which most values are zero or near-zero, so that...

Deep Learning

Spatial Pooling

Spatial pooling is a downsampling operation in convolutional neural networks (CNNs) that replaces a local region of a feature map with a single summary...

Computer VisionNeural Networks

Speculative Decoding

See also: transformer, large language model, inference, KV cache, autoregressive model, knowledge distillation Speculative decoding is a lossless inference...

AI InferenceDeep Learning

Speech recognition

Speech recognition, also known as automatic speech recognition (ASR), is the technology that converts spoken language into written text. It sits at the...

Deep LearningNatural Language Processing

SpinQuant

SpinQuant is a post-training quantization method for large language models that inserts learned rotation matrices into a transformer network to make its...

AI Infrastructure

Split

See also: Machine learning terms A split in machine learning is a partitioning operation, and the word names two distinct things. As a data operation, a split...

Splitter

A splitter is a term used in two distinct senses in machine learning. The first and most common sense is a utility that partitions a dataset into subsets such...

Model Evaluation

Squared Hinge Loss

Squared hinge loss (also called L2 hinge loss or L2-loss) is a loss function used in machine learning for classification tasks, most commonly in support vector...

Training & Optimization

Squared Loss

Squared loss, also called quadratic loss, L2 loss, or squared error loss, is a loss function that penalizes a prediction by the square of its error: for a true...

StatisticsTraining & Optimization

Stability

See also: Machine learning terms and Stability AI Stability in machine learning is the property that a learning algorithm or trained model produces similar...

State (Reinforcement Learning)

In reinforcement learning (RL), a state is a complete description of the environment at a particular point in time, containing all the information an agent...

Reinforcement Learning

State space model (deep learning)

A state space model (SSM) in deep learning is a class of sequence model that maps an input sequence to an output sequence through a fixed-size latent state,...

Deep Learning

State-Action Value Function

The state-action value function, written Q^π(s, a) and also called the action-value function or Q-function, gives the expected discounted return an agent...

Reinforcement Learning

Static

In machine learning, static means offline (also called batch): the model is trained once on a fixed dataset, or its predictions are computed in advance and...

Stationarity

Stationarity is a property of a time series or stochastic process whose statistical characteristics, such as the mean, variance, and autocovariance, do not...

Statistics

Statistical learning theory

Statistical learning theory (SLT) is the mathematical framework that explains when and why machine learning algorithms generalize from a finite training sample...

MathematicsStatistics

Stefano Ermon

Stefano Ermon is an Italian computer scientist and an associate professor of computer science at Stanford University, best known for foundational work on...

Generative AIPeople

Stochastic Gradient Descent (SGD)

See also: Machine learning terms Stochastic gradient descent (SGD) is an iterative optimization algorithm that updates model parameters using the gradient of...

Deep LearningTraining & Optimization

Stride

Stride is the step size by which a filter (or pooling window) moves across the input in a convolutional neural network (CNN): a stride of 1 shifts the filter...

Computer VisionDeep Learning

Structural risk minimization (SRM)

Structural risk minimization (SRM) is an inductive principle in statistical learning theory for selecting a learned model that simultaneously fits the training...

Subsampling

Subsampling is the practice of drawing a smaller subset from a larger collection of data points, training examples, features, or signal values, in order to cut...

Statistics

Summary

In TensorFlow, a summary is a piece of data written to disk during training so that it can later be visualized in TensorBoard. The tf.summary module is the API...

Supervised Learning

See also: Machine learning, Unsupervised learning, Reinforcement learning Supervised learning is a type of machine learning in which an algorithm learns from...

Artificial Intelligence

Supervised Machine Learning

See also: Machine learning terms Supervised machine learning is a branch of machine learning in which a model learns a mapping from inputs to outputs by...

Supervised fine-tuning

Supervised fine-tuning (SFT) is the process of further training a pre-trained large language model (LLM) on a smaller, labeled dataset of input-output pairs so...

Artificial IntelligenceDeep Learning

Support Vector Machine (SVM)

A support vector machine (SVM) is a supervised learning algorithm that classifies data by finding the optimal hyperplane that separates points into distinct...

Synthetic Feature

A synthetic feature (also called a constructed feature or derived feature) is a variable created by transforming, combining, or otherwise manipulating one or...

Data & DatasetsData Science

TIES-Merging

TIES-Merging is a training-free model merging method that combines several models fine-tuned from a shared pre-trained checkpoint into one multitask model...

Reinforcement Learning

TPU Board

A TPU board (Tensor Processing Unit board) is a printed circuit board (PCB) that houses one or more Tensor Processing Unit chips along with associated memory,...

AI HardwareGoogle

TPU Chip

This article covers the TPU chip and its hardware. For the broader topic, see Tensor Processing Unit (TPU). The Tensor Processing Unit (TPU) is a custom...

AI HardwareGoogle

TPU Device

A TPU device is a Google-designed application-specific integrated circuit (ASIC), the physical Tensor Processing Unit chip and its host hardware, built to...

AI HardwareGoogle

TPU Master

The TPU master is the control and coordination role in Google's Tensor Processing Unit (TPU) system that drives the actual TPU computation. In the TensorFlow...

AI HardwareGoogle

TPU Node

A TPU node is the legacy Google Cloud architecture for accessing Tensor Processing Unit (TPU) hardware, in which a user's virtual machine (VM) runs application...

AI HardwareAI Infrastructure

TPU Pod

A TPU Pod is a single Google supercomputer built from many Tensor Processing Unit (TPU) chips wired directly to each other by a high-speed Inter-Chip...

AI HardwareAI Infrastructure

TPU Slice

A TPU slice is a collection of Tensor Processing Unit (TPU) chips that all sit inside the same Google Cloud TPU Pod and are connected to one another by a...

AI HardwareGoogle

TPU Type

The types of TPU (Tensor Processing Unit) are the successive generations of custom machine-learning accelerator that Google has built since 2015: v1, v2, v3,...

AI HardwareGoogle

TPU Worker

A TPU worker is a virtual machine (VM) running Linux that has direct access to one or more Tensor Processing Unit (TPU) chips and executes the actual TPU...

AI HardwareAI Infrastructure

TPU resource

A TPU resource is an allocation of Tensor Processing Unit compute, Google's custom machine learning accelerator chips, that you reserve and run as a unit, most...

Tabular Classification Models

Part of the Tabular Models hub. See also the sibling survey Tabular Regression Models. Tabular classification models are machine learning systems that predict...

AI Models

Tabular Q-Learning

Tabular Q-learning is the classic form of Q-learning, a model-free reinforcement learning algorithm that stores the action-value function Q(s, a) explicitly in...

Reinforcement Learning

Tabular Regression Models

See also: Tabular Models and Tasks Tabular regression models are machine learning systems that predict a continuous numeric target from a vector of tabular...

AI Models

Tabular models

Tabular models are machine learning systems that learn from data arranged in tables, where each row is a sample and each column is a feature. The defining...

Data Science

Target

See also: Machine learning terms In supervised learning, the target is the variable that a model learns to predict from input features. It is the answer side...

Target Network

A target network is a separate, slowly updated copy of a neural network used in deep reinforcement learning to compute stable learning targets, decoupling the...

Deep LearningReinforcement Learning

Task arithmetic

Task arithmetic is a model-editing technique that steers the behavior of a neural network by adding or subtracting vectors in its weight space. The central...

Reinforcement Learning

Temperature (artificial intelligence)

Temperature is a hyperparameter that controls the randomness of a large language model's output by scaling the model's raw scores, called logits, before they...

Large Language ModelsNatural Language Processing

Temporal data

See also: Machine learning terms Temporal data is data where each observation is tagged with a timestamp, so the order in which observations arrive carries...

Statistics

Temporal-difference learning

Temporal-difference (TD) learning is a class of model-free reinforcement learning methods that learn value-function estimates by bootstrapping: updating each...

AlgorithmsReinforcement Learning

Tensor

In machine learning, a tensor is a multi-dimensional array of numbers that serves as the fundamental data structure for representing and manipulating data....

Deep LearningMathematics

Tensor Rank

The rank of a tensor, also referred to as its order or degree, is the number of dimensions (axes or indices) needed to describe the tensor. A scalar has rank...

Mathematics

Tensor Shape

A tensor shape is a tuple of integers that describes the number of elements along each dimension (or axis) of a tensor. For a tensor of rank r, the shape is...

Deep LearningMathematics

Tensor size

The size of a tensor is a description of how big the tensor is, and the term carries two distinct meanings in everyday deep learning. In its most precise...