Machine Learning

Squared Hinge Loss

Squared hinge loss (also called L2 hinge loss or L2-loss) is a loss function used in machine learning for classification tasks, most commonly in support vector machines (SVMs).

Training & Optimization

Squared Loss

Squared loss, also called quadratic loss, L2 loss, or squared error loss, is a loss function that penalizes a prediction by the square of its error: for a true value $y$ and a prediction $\hat{y}$

StatisticsTraining & Optimization

Stability

Stability in machine learning is the property that a learning algorithm or trained model produces similar outputs, whether predictions, parameter values, or loss curves, when its inputs are perturbed slightly.

State (Reinforcement Learning)

In reinforcement learning (RL), a state is a complete description of the environment at a particular point in time, containing all the information an agent needs to choose its next action.

Reinforcement Learning

State space model (deep learning)

A state space model (SSM) in deep learning is a class of sequence model that maps an input sequence to an output sequence through a fixed-size latent state

Deep Learning

State-Action Value Function

The state-action value function, written Q^π(s, a) and also called the action-value function or Q-function, gives the expected discounted return an agent obtains by taking action a in state s and thereafter…

Reinforcement Learning

Static

In machine learning, static means offline (also called batch): the model is trained once on a fixed dataset, or its predictions are computed in advance and cached, rather than being updated or generated…

Stationarity

Stationarity is a property of a time series or stochastic process whose statistical characteristics, such as the mean, variance, and autocovariance, do not change when the series is shifted in time.

Statistics

Statistical learning theory

Statistical learning theory (SLT) is the mathematical framework that explains when and why machine learning algorithms generalize from a finite training sample to unseen data.

MathematicsStatistics

Stefano Ermon

Stefano Ermon is an Italian computer scientist and an associate professor of computer science at Stanford University, best known for foundational work on score-based generative models

Generative AIPeople

Stochastic Gradient Descent (SGD)

Stochastic gradient descent (SGD) is a first-order optimization method that updates parameters using a gradient estimate computed from a randomly selected example or subset of examples.

Deep LearningTraining & Optimization

Stride

Stride is the step size by which a filter (or pooling window) moves across the input in a convolutional neural network (CNN): a stride of 1 shifts the filter one position at a time and visits every location…

Computer VisionDeep Learning

Structural risk minimization (SRM)

Structural risk minimization (SRM) is an inductive principle in statistical learning theory for selecting a learned model that simultaneously fits the training data and controls the capacity of the hypothesis…

Subsampling

Subsampling is the practice of drawing a smaller subset from a larger collection of data points, training examples, features, or signal values, in order to cut compute cost, add regularization, or rebalance a…

Statistics

Summary

In TensorFlow, a summary is a piece of data written to disk during training so that it can later be visualized in TensorBoard. The tf.summary module is the API used to record those values.

Supervised Learning

Supervised learning is a machine learning paradigm in which a model is fitted to examples that pair an input with an observed target.

Artificial Intelligence

Supervised Machine Learning

Supervised machine learning is a branch of machine learning in which a model learns a mapping from inputs to outputs by training on labeled examples, where every input (a feature vector) is paired with the…

Supervised fine-tuning

Supervised fine-tuning (SFT) is supervised training applied to a model that has already been trained.

Artificial IntelligenceDeep Learning

Support Vector Machine (SVM)

A support vector machine (SVM) is a family of supervised learning methods used for classification and regression.

Synthetic Feature

A synthetic feature (also called a constructed feature or derived feature) is a variable created by transforming, combining, or otherwise manipulating one or more existing features in a dataset, rather than…

Data & DatasetsData Science

TIES-Merging

TIES-Merging is a training-free model merging method that combines several models fine-tuned from a shared pre-trained checkpoint into one multitask model while explicitly resolving the interference between…

Reinforcement Learning

TPU Board

A TPU board (Tensor Processing Unit board) is a printed circuit board (PCB) that houses one or more Tensor Processing Unit chips along with associated memory, power delivery, and interconnect components.

AI HardwareGoogle

TPU Chip

This article covers the TPU chip and its hardware. For the broader topic, see Tensor Processing Unit (TPU).

AI HardwareGoogle

TPU Device

A TPU device is a Google-designed application-specific integrated circuit (ASIC), the physical Tensor Processing Unit chip and its host hardware

AI HardwareGoogle

TPU Master

The TPU master is the control and coordination role in Google's Tensor Processing Unit (TPU) system that drives the actual TPU computation.

AI HardwareGoogle

TPU Node

A TPU node is the legacy Google Cloud architecture for accessing Tensor Processing Unit (TPU) hardware, in which a user's virtual machine (VM) runs application code and communicates with a separate

AI HardwareAI Infrastructure

TPU Pod

A TPU Pod is a single Google supercomputer built from many Tensor Processing Unit (TPU) chips wired directly to each other by a high-speed Inter-Chip Interconnect (ICI) fabric arranged as a 2D or 3D torus, so…

AI HardwareAI Infrastructure

TPU Slice

A TPU slice is a collection of Tensor Processing Unit (TPU) chips that all sit inside the same Google Cloud TPU Pod and are connected to one another by a high-speed inter-chip interconnect (ICI).

AI HardwareGoogle

TPU Type

The types of TPU (Tensor Processing Unit) are the successive generations of custom machine-learning accelerator that Google has built since 2015: v1, v2, v3, v4, v5e, v5p, v6e (Trillium), and v7 (Ironwood).

AI HardwareGoogle

TPU Worker

A TPU worker is a virtual machine (VM) running Linux that has direct access to one or more Tensor Processing Unit (TPU) chips and executes the actual TPU computation on that attached hardware.

AI HardwareAI Infrastructure

TPU resource

A TPU resource is an allocation of Tensor Processing Unit compute, Google's custom machine learning accelerator chips, that you reserve and run as a unit

Tabular Classification Models

Tabular classification models predict a discrete label, or a probability distribution over labels, from rows of a table.

AI Models

Tabular Q-Learning

Tabular Q-learning is the classic form of Q-learning, a model-free reinforcement learning algorithm that stores the action-value function Q(s, a) explicitly in a lookup table, the Q-table

Reinforcement Learning

Tabular Regression Models

Tabular regression models are machine learning systems that predict a continuous numeric target from a vector of tabular features, where rows are samples and columns are heterogeneous attributes (numeric…

AI Models

Tabular models

Tabular models are machine learning systems that learn from data arranged in tables, where each row is a sample and each column is a feature.

Data Science

Target

In supervised learning, the target is the variable that a model learns to predict from input features.

Target Network

A target network is a separate, slowly updated copy of a neural network used in deep reinforcement learning to compute stable learning targets

Deep LearningReinforcement Learning

Task arithmetic

Task arithmetic is a model-editing technique that steers the behavior of a neural network by adding or subtracting vectors in its weight space.

Reinforcement Learning

Temperature (artificial intelligence)

Temperature is a hyperparameter that controls the randomness of a large language model's output by scaling the model's raw scores, called logits

Large Language ModelsNatural Language Processing

Temporal data

Temporal data is data where each observation is tagged with a timestamp, so the order in which observations arrive carries meaning.

Statistics

Temporal-difference learning

Temporal-difference (TD) learning is a class of model-free reinforcement learning methods that learn value-function estimates by bootstrapping: updating each estimate of how good a state is toward a target…

AlgorithmsReinforcement Learning

Tensor

In machine learning, a tensor is a multi-dimensional array of numbers that serves as the fundamental data structure for representing and manipulating data.

Deep LearningMathematics

Tensor Rank

The rank of a tensor, also referred to as its order or degree, is the number of dimensions (axes or indices) needed to describe the tensor.

Mathematics

Tensor Shape

A tensor shape is a tuple of integers that describes the number of elements along each dimension (or axis) of a tensor.

Deep LearningMathematics

Tensor size

The size of a tensor is a description of how big the tensor is, and the term carries two distinct meanings in everyday deep learning.

TensorBoard

TensorBoard is the open-source visualization toolkit for TensorFlow, described by Google as "a suite of visualization tools to understand, debug, and optimize TensorFlow programs for ML experimentation." It…

Data ScienceDeveloper Tools

TensorFlow

TensorFlow is an open-source software platform for numerical computation and machine learning.

AI Tools & ProductsDeep Learning

TensorFlow Decision Forests (TF-DF)

TensorFlow Decision Forests (often abbreviated TF-DF) is an open-source Google library for training, serving, and interpreting decision-forest models such as Random Forest and Gradient Boosted Decision Trees…

Developer ToolsGoogle

TensorRT

TensorRT is NVIDIA's software development kit (SDK) for high-performance deep learning inference on NVIDIA GPUs.

AI HardwareAI Tools & Products

Termination condition

A termination condition, also called a stopping criterion, convergence criterion, or halting condition, is a rule that decides when an iterative algorithm should stop running.

Training & Optimization

Test Set

A test set is a collection of examples reserved from model fitting and model selection so that it can evaluate a fixed machine learning model or a fully specified learning procedure.

Model Evaluation

Test loss

Test loss is the value of a loss function computed on a held-out test data set: data that was used neither for training nor for validation or model selection.

Test-Time Training (TTT)

Test-Time Training (TTT) is a family of machine learning techniques in which a model updates a subset of its own parameters at inference time

AI InferenceTraining & Optimization

Test-time compute

Test-time compute (also called inference-time compute scaling or test-time scaling) is the practice of allocating additional computation while a large language model answers a query

Artificial IntelligenceLarge Language Models

Text summarization

Text summarization is the natural language processing (NLP) task of automatically producing a shorter version of one or more documents that preserves the most important information from the original text.

Deep LearningNatural Language Processing

The Pile (dataset)

The Pile is an 825.18 GiB (approximately 886 GB) English text corpus designed for training large language models, assembled from 22 diverse, high-quality subsets spanning academic, professional, internet…

Data & DatasetsNatural Language Processing

The Stack v2

The Stack v2 is a large open dataset of source code released by BigCode in February 2024 as the training dataset behind the StarCoder2 family of code models.

Data & Datasets

Threshold (for decision trees)

In a decision tree, a threshold is the cut point used in an internal node's split test that decides which child subtree a sample is routed to.

TikTok

TikTok is a short-form video app owned by the Chinese technology company ByteDance, best known in artificial intelligence circles as the most widely studied production recommender system: its For You Page…

AI Policy & RegulationAI Tools & Products

Time Series

A time series is a sequence of observations recorded in time order, often at regular intervals: daily retail sales, hourly electricity demand, per-minute server latency, quarterly GDP.

Data ScienceDeep Learning