Machine learning terms
Machine learning terms are the vocabulary used to describe data, models, training procedures, evaluation methods, and deployed behavior. This page is a curated glossary of terms that recur across research papers, software documentation, and applied practice. It is a companion to the broader Machine Learning article, not a second survey of the field. Readers who need an exhaustive index can use Machine learning terms/All; readers who need a tutorial or the history of a method should follow the linked concept article.
The vocabulary is not controlled by one universal glossary. International terminology standards, public-sector glossaries, textbooks, and framework documentation overlap, but they serve different audiences and sometimes draw boundaries differently.[1][2][3] A model, for example, may mean a fitted mathematical function, a set of learned parameters, or a packaged artifact. Bias can refer to a statistical error component, a parameter in a neural network, or a harmful sociotechnical tendency. Token, state, fairness, and explainability likewise depend on the task and discipline. The definitions below state the intended context and preserve important qualifications rather than forcing all uses into one meaning.
Most entries describe concepts, not products. A term may have a dedicated AI Wiki page even when its definition here is intentionally short. Classification metrics assume that the positive class and any averaging rule have been specified. Probabilistic metrics depend on the evaluation population and data-generating conditions. Reinforcement-learning entries distinguish an environment's state from the observation available to an agent, and an immediate reward from the return optimized over time. Fairness criteria describe mathematical constraints, not complete judgments about legality or justice. These conventions make the glossary useful as a navigation aid while keeping the separate specialist glossaries and full articles responsible for deeper treatment.
Scope and conventions
This glossary favors durable concepts over changing library interfaces, cloud product names, or model rosters. A definition describes common technical usage unless it explicitly identifies a narrower statistical, legal, or reinforcement-learning meaning. Symbols and formulas are omitted where ordinary language is sufficient; the linked pages provide derivations and implementation details.
| Convention | How to read an entry |
|---|---|
| Classification metrics | The positive class, decision threshold, sample weighting, and multiclass or multilabel averaging rule must be specified before two reported values are comparable.[7] |
| Probabilities | A score interpreted as a probability should be checked for calibration on a relevant evaluation population, not assumed to be calibrated because it lies between zero and one.[8] |
| Data splits | Training data fit parameters, validation data support choices such as hyperparameters, and a final test set estimates performance only while it remains isolated from those choices.[6][9] |
| Fairness | Group metrics formalize different and sometimes incompatible criteria. They do not by themselves determine whether a system is fair, lawful, or appropriate for a use.[14][15] |
| Reinforcement learning | Reward is an immediate signal, return aggregates future rewards, state is a property of the process model, and an observation may reveal only part of that state.[10] |
| Language models | Token counts and perplexity depend on tokenization. Comparisons require the same data, units, and evaluation convention.[12] |
Topic guides
The focused glossaries organize specialized vocabulary without duplicating every definition here.
| Topic guide | Coverage |
|---|---|
| Fundamentals | Data, examples, features, labels, training, optimization, generalization, and basic evaluation |
| Natural language processing | Tokens, language models, sequence tasks, embeddings, generation, and language evaluation |
| Fairness | Group criteria, bias analysis, sensitive attributes, and mitigation vocabulary |
| Decision forests | Trees, bagging, random forests, boosting, splits, and feature importance |
| Reinforcement learning | Agents, environments, policies, value functions, rewards, and sequential decisions |
| Computer vision | Images, convolutions, detection, segmentation, bounding boxes, and overlap measures |
| Sequence models | Recurrent networks, attention, transformers, and ordered data |
| Clustering | Unsupervised grouping, centroids, similarity, and hierarchical methods |
| Recommendation systems | Candidate generation, ranking, collaborative filtering, and matrix factorization |
| TensorFlow | Framework-specific tensors, graphs, layers, training, and deployment terms |
| Google Cloud | Product-specific infrastructure and managed machine-learning vocabulary |
Core glossary
A-C
| Term | Definition |
|---|---|
| Accuracy | The proportion of evaluated predictions that match their reference labels. It can conceal poor performance on a rare or high-cost class, so the class distribution and complementary metrics matter.[7] |
| Action | A choice available to an agent. In a reinforcement-learning model, an action can affect the next state, the reward, or both according to the environment's dynamics.[10] |
| Activation function | A function applied to a unit's pre-activation, often to introduce nonlinearity. Different choices also affect gradient flow, output range, and optimization behavior.[5] |
| Agent | The decision-making entity in a reinforcement-learning formulation. It receives observations or states, selects actions under a policy, and experiences consequences supplied by an environment.[10] |
| Attention | A mechanism that forms context-dependent combinations of value representations using weights derived from queries and keys. Attention can be used inside recurrent, convolutional, or transformer systems.[5][11] |
| AUC | Usually the area under a receiver operating characteristic curve. For binary scores it summarizes ranking across thresholds; its probabilistic ranking interpretation depends on conventions for direction and tied scores.[7] |
| Autoencoder | A model trained to encode an input into a representation and decode it toward a reconstruction. An ordinary autoencoder is not automatically a probabilistic generative model.[5] |
| Backpropagation | An efficient application of the chain rule that propagates derivatives backward through a computational graph. It computes gradients; a separate optimizer decides how parameters change.[5] |
| Bagging | An ensemble procedure that fits learners to resampled training data and aggregates their predictions. Bootstrap aggregation can reduce variance when the component learners are sufficiently diverse.[4] |
| Batch | A collection of examples processed together for one forward evaluation or training step. A batch may be the entire dataset or a subset of it.[3][5] |
| Batch normalization | A layer that standardizes selected activations using batch statistics during training, then applies learned scale and offset parameters. Inference commonly uses stored estimates rather than the current batch.[5] |
| Batch size | The number of examples in a batch. It influences memory use, the noise of a gradient estimate, throughput, and sometimes optimization behavior.[3][5] |
| Bellman equation | A recursive relation that expresses a value function through expected immediate reward and the value of successor states or state-action pairs.[10] |
| Bias-variance tradeoff | A way to analyze prediction error in which overly restrictive models can have high systematic error while highly flexible fitted models can vary strongly across training samples. It is not a universal two-number diagnosis for every learning problem.[4] |
| Binary classification | A classification problem whose target has two classes. Metrics still require an explicit choice of which class is treated as positive.[7] |
| Boosting | A family of ensemble methods that builds learners sequentially so later stages address shortcomings of the current ensemble. Reweighting misclassified examples is one form, not the definition of all boosting.[4] |
| Bounding box | A rectangular region used to localize an object or area of interest in an image. Coordinate conventions and whether edges are inclusive must be specified when boxes are exchanged or scored.[3] |
| Calibration | Agreement between predicted probabilities and observed frequencies in a defined population. For example, among cases assigned probability near 0.8, a calibrated binary predictor is positive about 80 percent of the time.[8] |
| Class | One of the discrete target categories in a classification task. A class is a semantic or operational category, while its stored label may be a number or string.[3] |
| Classification | Supervised prediction of discrete labels or label sets. Binary, multiclass, multilabel, and ordinal tasks require different output and evaluation conventions.[4][7] |
| Clustering | Unsupervised organization of examples into groups according to a chosen representation, similarity measure, and algorithm. A cluster is therefore method-dependent rather than an automatically discovered natural category.[4] |
| Collaborative filtering | Recommendation from patterns of interactions among users and items, rather than only from item attributes. Sparse feedback and unseen users or items create characteristic cold-start limitations.[3] |
| Computer vision | The study and construction of systems that extract, predict, or generate information involving images or video. Tasks include classification, detection, segmentation, tracking, and generation.[3][5] |
| Confusion matrix | A table whose rows and columns cross reference labels with predicted labels. Its orientation must be stated because software packages differ over which axis represents truth.[7] |
| Convolution | An operation that combines a local kernel with an input across positions while sharing parameters. Deep-learning libraries often implement cross-correlation while retaining the conventional name convolution.[5] |
| Convolutional neural network | A neural network that uses convolutional layers to exploit local structure and parameter sharing. Convolutional networks are common in vision but also apply to signals, sequences, and grids.[5] |
| Cross-entropy | The expected negative log probability that one distribution assigns to outcomes drawn from a target distribution. With one-hot classification targets, it becomes the negative log probability assigned to the reference class.[5] |
| Cross-validation | A resampling procedure that rotates observations through training and validation folds to estimate or compare models. Preprocessing and model selection must occur inside each training fold to avoid leakage.[6][9] |
D-F
| Term | Definition |
|---|---|
| Data augmentation | Generation of additional training examples by applying label-preserving or target-adjusting transformations to existing data. Whether a transformation is valid depends on the task, such as rotation for some images but not all.[3][5] |
| Data leakage | Use during model construction of information that would not legitimately be available when predictions are made. Leakage usually produces an overly optimistic evaluation and can occur through preprocessing, target proxies, time, or split contamination.[9] |
| Dataset | A defined collection of examples and associated metadata used for training, validation, testing, or analysis. Its sampling process, provenance, schema, and intended use are part of its meaning.[3] |
| Decision boundary | The set of input points at which a classifier changes its predicted class or decision. The boundary depends on the representation, model, parameters, and decision rule.[3] |
| Decision tree | A model that routes an example through successive feature-based splits to a leaf prediction. Trees can be used for classification or regression and are often components of ensembles.[4] |
| Deep learning | Machine learning based on layered representations whose intermediate features are learned from data. Depth is relative to the architecture and problem; there is no universal minimum hidden-layer count.[5] |
| Deep neural network | A neural network with multiple successive representation-learning stages. The word deep describes architectural composition, not a fixed threshold of exactly two or any other number of hidden layers.[5] |
| Deep Q-network | A Q-learning system that approximates action values with a deep neural network. The influential DQN design combined this approximation with experience replay and a separately updated target network.[10] |
| Demographic parity | A group-fairness criterion under which the rate of a selected prediction or decision is equal across specified groups. It does not condition on the reference outcome and may conflict with other criteria.[14][15] |
| Diffusion model | A generative model trained around a gradual noising process and a learned reverse or denoising process. Sampling typically begins from noise and follows repeated learned transitions toward a data example.[18] |
| Dimensionality reduction | Transformation of data into fewer coordinates while attempting to preserve selected structure, information, or predictive utility. Different methods preserve different properties.[4] |
| Discriminator | In a generative adversarial network, the model trained to distinguish samples from the data distribution from samples produced by the generator.[16] |
| Disparate impact | Disproportionately adverse effects on a protected group from an apparently neutral practice. It has jurisdiction-specific legal meanings, so a statistical disparity alone is not a complete legal determination.[15] |
| Distribution shift | A change between the data distribution used to build or evaluate a model and the distribution encountered later. The shift may affect inputs, targets, their relationship, or the surrounding decision process.[3] |
| Dropout | A training method that randomly omits selected units or connections during updates. Inference uses a deterministic full network under a scaling convention that accounts for the training-time omissions.[5] |
| Early stopping | Stopping optimization according to performance on held-out validation data rather than continuing until training loss is minimized. Repeated choices based on the same validation set can themselves overfit it.[5][6] |
| Embedding | A vector representation assigned to an item, token, category, or larger object so that a model can operate on it. The geometry is learned or constructed for a task and does not have one context-free semantic interpretation.[3][12] |
| Ensemble | A predictor formed by combining outputs from multiple fitted models. Averaging, voting, stacking, bagging, and boosting create ensembles with different assumptions and failure modes.[4] |
| Environment | Everything in a reinforcement-learning formulation outside the agent that receives actions and produces subsequent states or observations and rewards.[10] |
| Epoch | One traversal of the designated training examples, under a stated sampling convention. With resampling, streaming data, or distributed training, implementations may define an epoch operationally rather than as one unique view of every example.[3] |
| Equality of opportunity | The one-sided equalized-odds criterion proposed for supervised learning, usually requiring equal true-positive rates across protected groups for the favorable reference outcome.[14] |
| Equalized odds | A group-fairness criterion requiring prediction to be independent of the protected group conditional on the reference outcome. In binary classification this corresponds to equal true-positive and false-positive rates across groups.[14] |
| Example | One instance presented to or evaluated by a learning procedure. It contains input features and, in supervised settings, may also contain one or more targets.[3] |
| Explainability | The extent to which a system or supporting method can provide information people can use to understand a result or behavior. A plausible explanation is not automatically a faithful account of the model's computation.[2] |
| Exploding gradient | A training failure in which derivatives grow to very large magnitudes as they propagate through a computation, causing unstable parameter updates or numerical overflow.[5] |
| F1 score | The harmonic mean of precision and recall for a specified positive class. It ignores true negatives and does not encode all application costs, probability calibration, or threshold preferences.[7] |
| Fairness metric | A quantitative measure of a specified fairness-related property, often a comparison across groups. Selecting a metric is a normative and contextual choice; satisfying it does not establish fairness in every other sense.[14][15] |
| Feature | A measured, derived, or encoded input variable supplied to a model. A feature can be numeric, categorical, structured, or represented by multiple coordinates.[3][4] |
| Feature engineering | Selection, construction, encoding, or transformation of raw observations into model inputs. The transformations must be fit without access to held-out outcomes that would create leakage.[4][9] |
| Fine-tuning | Additional training of a pretrained model for a new dataset, domain, task, or behavior. Fine-tuning may update all parameters or only a selected subset or added parameters.[3][13] |
| Foundation model | A model trained on broad data, usually at scale, that can be adapted to many downstream tasks. The category concerns broad upstream training and downstream adaptation, not one architecture or a guarantee of general competence.[13] |
G-L
| Term | Definition |
|---|---|
| Generalization | Performance on relevant examples not used to fit the model. A generalization claim is bounded by the population, task, time period, and evaluation procedure from which it is estimated.[4] |
| Generative adversarial network | A generative framework in which a generator and discriminator are trained in opposition: the generator tries to imitate the data distribution while the discriminator tries to distinguish generated from observed samples.[16] |
| Generative model | A model of a data distribution, joint distribution, or conditional distribution that can support generation or likelihood-related tasks. Some generative models define an explicit density and others generate samples only implicitly.[5] |
| Generator | In a generative adversarial network, the model that transforms noise or another input into candidate samples intended to resemble data from the target distribution.[16] |
| Gradient | The vector of partial derivatives of a scalar quantity with respect to selected variables. In training, gradients describe local sensitivity of an objective to parameter changes.[5] |
| Gradient boosting | A boosting method that adds weak learners in a stagewise fashion to move the ensemble toward lower loss, commonly by fitting directions related to the negative functional gradient.[4] |
| Gradient descent | An iterative optimization family that changes parameters in the direction opposite the local gradient of an objective. Step size, gradient estimation, curvature, and nonconvexity affect its behavior.[4][5] |
| Hallucination | Generated content that is nonsensical, unfaithful to provided source content, or inconsistent with relevant facts. Operational definitions differ across generation tasks and must be paired with an evaluation protocol.[3][12][19] |
| Hidden layer | A neural-network layer between the externally supplied input and the designated output. Hidden activations form intermediate representations rather than directly observed target values.[5] |
| Hyperparameter | A setting that governs model structure, data processing, or learning and is not fitted as an ordinary model parameter in the same training run. Some hyperparameters are selected through a separate optimization process.[3][4] |
| Hyperparameter tuning | Search or optimization over hyperparameter choices using training and validation evidence. The untouched test set should not guide this search.[6] |
| Inference | Use of a fitted model to compute predictions, scores, representations, or generated outputs. In statistics the word can also mean reasoning about populations or latent quantities, so context matters.[1][3] |
| Intersection over union | The area or volume of overlap between two regions divided by the area or volume covered by their union. It is commonly used to compare predicted and reference boxes or masks.[3] |
| K-means | A clustering algorithm that alternates assignment to the nearest centroid with recomputation of centroids to reduce within-cluster squared Euclidean distance. Results can depend on initialization and feature scaling.[4] |
| Label | A stored target value associated with an example, especially in supervised learning. Labels can be noisy, incomplete, delayed, subjective, or produced by another measurement process.[2][3] |
| Language model | A model that assigns probabilities to linguistic sequences or predicts tokens conditional on context. The unit may be a character, subword, word, byte, or another token defined by the system.[12] |
| Large language model | A language model with large learned capacity and broad training data relative to a chosen comparison set. There is no universal parameter-count threshold at which a language model becomes large.[12][13] |
| Learning rate | A hyperparameter that scales an optimizer's update or a component of it. Adaptive optimizers may maintain different effective rates across parameters and training steps.[3][5] |
| Log loss | Negative log-likelihood used to score probabilistic predictions. In binary classification it penalizes confident probability assignments sharply when they disagree with the reference label.[7] |
| Logistic regression | A generalized linear classification model. In the binary case, it represents class log-odds as a linear function of features; multinomial extensions model more than two classes.[3][4] |
| Long short-term memory | A gated recurrent architecture whose cell state and gates create paths that can preserve or modify information over many sequence steps. It mitigates, but does not eliminate, long-range optimization problems.[5] |
| Loss | A numerical penalty assigned to a prediction, example, or collection of examples. The training objective may aggregate the loss and add regularization or other constraints, so loss and objective are not always synonyms.[3][5] |
M-R
| Term | Definition |
|---|---|
| Machine learning | The development or use of computational methods that improve task performance or infer patterns from data or experience. The boundary with statistics, optimization, artificial intelligence, and rule-based programming depends on context.[1][3][4] |
| Markov decision process | A mathematical model of sequential decisions with states, actions, transition probabilities, rewards, and usually a discount or horizon, where the modeled state is sufficient for predicting the next-step distribution given an action.[10] |
| Matrix factorization | Approximation or decomposition of a matrix as a product of lower-dimensional factors. In recommendation, user and item factors can model interaction patterns but do not by themselves solve cold-start or feedback-bias problems.[3] |
| Mini-batch | A subset of training examples used to estimate a gradient or perform one update. Mini-batch training lies between single-example stochastic updates and full-batch updates.[3][5] |
| Model | A fitted mathematical or computational object that maps inputs, represents a distribution, or supports decisions. The word can refer narrowly to learned parameters or more broadly to their architecture and execution logic.[1][2] |
| Multiclass classification | Classification in which one example is assigned among more than two classes. It differs from multilabel classification, where multiple labels can be true for the same example.[7] |
| Multimodal model | A model designed to consume, relate, or produce information in more than one modality, such as text, images, audio, or video. Multimodal does not imply equal capability in every modality.[13] |
| Natural language processing | Computational analysis or generation of human language and related symbolic sequences. It includes tasks involving text, speech transcripts, structure, meaning, dialogue, retrieval, and generation.[12] |
| Neural network | A parameterized composition of layers or computational units, commonly trained with gradient-based methods. Network structure, activation functions, objective, and data determine what function is learned.[5] |
| Objective function | The scalar quantity an optimization procedure seeks to minimize or maximize. It may combine empirical loss, regularization, constraints, rewards, or surrogate terms.[4][5] |
| Optimizer | The algorithm that turns gradients or other training signals into parameter updates. Its update rule, internal state, and hyperparameters are distinct from the loss being optimized.[5] |
| Overfitting | Learning patterns that produce strong performance on the fitting data but fail to generalize to the intended population. It is diagnosed with appropriate held-out evidence, not training error alone.[4] |
| Parameter | A value learned or estimated during model fitting, such as a weight, bias, split value, or distribution parameter. This contrasts with a hyperparameter set or selected outside the ordinary fit.[3][4] |
| Perplexity | The exponential of average negative log-likelihood under a specified unit and dataset, widely used for language models. Values are not directly comparable across different tokenizations, corpora, or normalization conventions.[12] |
| Policy | A reinforcement-learning rule that maps states, observations, or histories to actions or probability distributions over actions.[10] |
| Pooling | Aggregation over a local region or collection, such as maximum or average pooling. In convolutional networks it can reduce spatial resolution and create limited invariance.[5] |
| Positional encoding | Information supplied to a sequence model so it can distinguish positions or relative order. It may be fixed, learned, absolute, relative, or incorporated directly into attention.[11][12] |
| Precision | Among predictions assigned to a specified positive class, the fraction whose reference label is positive. Its value depends on the threshold and class prevalence in the evaluation data.[7] |
| Prediction | A model output intended to estimate an unknown target, future event, class, score, probability, or structured object. Not every output interpreted as a prediction is a calibrated probability.[2][8] |
| Pretraining | An initial training phase that produces representations or parameters for later adaptation, prompting, or task-specific learning. The pretraining objective need not match the downstream task.[12][13] |
| Prompt | Input supplied to condition a generative or instruction-following model, potentially including instructions, examples, context, tool descriptions, or requested output format.[3][12] |
| Q-learning | An off-policy temporal-difference method that learns action values toward a target using the best estimated next action, without requiring the behavior policy to be the target policy.[10] |
| Random forest | An ensemble that combines decision trees while randomizing the candidate features considered at splits. Trees are commonly, but not necessarily, trained on bootstrap samples; both mechanisms can reduce correlation among trees.[4] |
| Recall | Among reference examples in a specified positive class, the fraction predicted positive. Recall is also called sensitivity or true-positive rate in common binary-classification settings.[7] |
| Recommender system | A system that selects or ranks items for users or contexts using interaction, content, or other signals.[3] |
| Rectified linear unit | An activation that returns zero for a negative input and the input itself for a positive input. Its simple derivative supports optimization, though units can become inactive on consistently negative inputs.[5] |
| Recurrent neural network | A neural network that reuses parameters across sequence steps and updates a hidden representation from prior state and current input. Gated variants modify this recurrence to support longer dependencies.[5] |
| Regularization | Any training strategy intended to improve generalization by shaping the fitted solution, such as penalties, constraints, augmentation, noise, early stopping, or architectural choices. It is broader than adding a penalty term.[5] |
| Reinforcement learning | Learning how to act through interaction so as to maximize expected return. The formulation requires choices about observations or states, actions, rewards, dynamics, horizon, and policy evaluation.[10] |
| Representation | An encoding of relevant information used internally or exposed as model output. What a representation preserves depends on the data, objective, architecture, and analysis method.[5][13] |
| Return | The accumulated future reward from a time step under a specified horizon and discounting convention. Return, rather than each immediate reward in isolation, is the usual reinforcement-learning objective signal.[10] |
| Reward | The immediate scalar feedback emitted by a reinforcement-learning environment after a transition or action. A reward defines part of the task but may be an imperfect proxy for the intended outcome.[10] |
| ROC curve | A curve plotting true-positive rate against false-positive rate as a binary decision threshold varies. It summarizes score discrimination across thresholds, not probability calibration.[7][8] |
S-Z
| Term | Definition |
|---|---|
| Self-attention | Attention in which queries, keys, and values are derived from positions in the same sequence or representation. Masks can restrict which positions are allowed to interact.[11] |
| Semi-supervised learning | Learning from a combination of labeled and unlabeled examples. Its benefit depends on assumptions connecting the unlabeled-data structure to the prediction task.[3] |
| Sensitive attribute | A characteristic used to define groups for fairness, privacy, or discrimination analysis, such as age or race in some contexts. Which attributes are protected and how they may be used depend on jurisdiction and application.[15] |
| Sequence model | A model whose inputs, outputs, or internal dependencies are ordered. Recurrent networks, convolutional sequence models, state-space models, and transformers are different sequence-model families.[5][12] |
| Sequence-to-sequence task | A task mapping an input sequence to an output sequence whose length may differ, such as translation, transcription, or summarization.[12] |
| Softmax | A function that turns a vector of finite real-valued scores into positive values summing to one. Treating them as probabilities still depends on the model, objective, and calibration.[5][8] |
| State | In a Markov decision process, information sufficient to determine the distribution of the next state and reward given an action. An agent's observation can be an incomplete or noisy view of that state.[10] |
| Stochastic gradient descent | Gradient-based optimization using a noisy estimate computed from one example or a mini-batch rather than the complete training set. Usage often includes mini-batch SGD even though the literal cases differ.[4][5] |
| Supervised learning | Learning a mapping or predictive rule from examples paired with target values. Labels may be measured or annotated and are not automatically complete or error-free ground truth.[3][4] |
| Tensor | A multidimensional array in common machine-learning software usage. Its rank is the number of dimensions and its shape records the size along each dimension.[3] |
| Test set | Held-out examples reserved for final evaluation after model and hyperparameter choices. Reusing test results to guide development turns the set into another validation source.[6][9] |
| Token | A unit emitted by a tokenizer and consumed or predicted by a language model. It may represent a word, word piece, character, byte, punctuation mark, or another learned unit rather than the smallest linguistic element.[12] |
| Tokenization | Conversion of text or another input into a sequence of tokens under a defined vocabulary and algorithm. Tokenization choices affect sequence length, model inputs, and evaluation quantities such as perplexity.[12] |
| Training | The process of fitting model parameters or behavior from data, feedback, or interaction according to an objective and optimization procedure.[1][3] |
| Training set | The examples used to fit model parameters. Transformations learned from data, including scaling and feature selection, should be fit using the training portion rather than held-out data.[6][9] |
| Transfer learning | Reuse of parameters, representations, or knowledge learned in one setting to improve learning or performance in another. Transfer can help, fail, or harm when source and target differ.[3][13] |
| Transformer | A sequence architecture centered on attention, position information, feed-forward transformations, residual connections, and normalization. The original encoder-decoder design removed recurrence and convolution, while later transformer variants modify its components.[11] |
| Underfitting | Failure to capture enough task-relevant structure, producing inadequate performance even on data representative of the fitting objective. It can result from limited model capacity, features, optimization, or training.[4] |
| Unsupervised learning | Learning structure or representations from inputs without task labels of the supervised kind. Clustering, density estimation, dimensionality reduction, and some representation-learning methods fall under this broad heading.[3][4] |
| Validation set | Held-out data used during development for choices such as hyperparameters, stopping, thresholds, or model selection. Because those choices adapt to it, final performance should be estimated on separate test data.[6] |
| Vanishing gradient | A training difficulty in which derivatives shrink through many composed operations, leaving early parameters with little learning signal. Gating, residual paths, normalization, and activation choices can change the problem.[5] |
| Variational autoencoder | A latent-variable generative model trained with variational inference by optimizing a lower bound that balances reconstruction or likelihood fit against a divergence involving the approximate latent posterior.[17] |
| Weight | A learned coefficient that controls the contribution of an input or intermediate activation. Neural-network weights are parameters, but not every parameter is necessarily called a weight.[3][5] |
| Word embedding | A vector representation associated with a word or word type. Context-independent word embeddings assign one stored vector per vocabulary item, while contextual models produce representations that vary with surrounding text.[12] |
| Zero-shot learning | Performing a task or recognizing a class without labeled training examples specific to that task or class, typically by transferring semantic descriptions, instructions, or broadly learned representations. Exact usage varies by subfield.[3][13] |
Reading and maintaining the glossary
A short definition should be treated as an entry point, not as a substitute for task documentation. Before comparing metrics, identify the dataset, split, population, target, averaging rule, threshold, and uncertainty. Before comparing language models, also identify tokenization, prompt, decoding settings, tools, and evaluation protocol. Before interpreting a fairness result, state the protected groups, reference outcome, decision context, and criterion selected.[6][7][8][12][14][15]
Terminology changes when methods cross disciplines. The safest interpretation is the narrowest one supported by the surrounding paper, standard, or implementation. If an article uses a word differently, that local definition controls its argument. This page should therefore be updated when a definition becomes misleading or a durable term becomes common, but it should not become a changelog for frameworks or a leaderboard for current systems.
References
- ^International Organization for Standardization and International Electrotechnical Commission, "ISO/IEC 22989:2022 Information technology - Artificial intelligence - Artificial intelligence concepts and terminology," 2022. iso.org/...74296
- ^National Institute of Standards and Technology, "The Language of Trustworthy AI: An In-Depth Glossary of Terms," beta glossary, accessed 28 July 2026. airc.nist.gov/glossary
- ^Google for Developers, "Machine Learning Glossary," accessed 28 July 2026. developers.google.com/...glossary
- ^Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani, "An Introduction to Statistical Learning," second R edition, 2021; and Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, and Jonathan Taylor, Python edition, 2023, official book site. statlearning.com
- ^Ian Goodfellow, Yoshua Bengio, and Aaron Courville, "Deep Learning," MIT Press, 2016, official book site. deeplearningbook.org
- ^scikit-learn, "Cross-validation: evaluating estimator performance," version 1.9.0 documentation, accessed 28 July 2026. scikit-learn.org/...cross_validation
- ^scikit-learn, "Metrics and scoring: quantifying the quality of predictions," version 1.9.0 documentation, accessed 28 July 2026. scikit-learn.org/...model_evaluation
- ^scikit-learn, "Probability calibration," version 1.9.0 documentation, accessed 28 July 2026. scikit-learn.org/...calibration
- ^scikit-learn, "Common pitfalls and recommended practices," version 1.9.0 documentation, accessed 28 July 2026. scikit-learn.org/...common_pitfalls
- ^Richard S. Sutton and Andrew G. Barto, "Reinforcement Learning: An Introduction," second edition, MIT Press, 2018. mitpress.mit.edu/...reinforcement-learning
- ^Ashish Vaswani et al., "Attention Is All You Need," 2017. arxiv.org/...1706.03762
- ^Daniel Jurafsky and James H. Martin, "Speech and Language Processing," third edition draft, online manuscript released 6 January 2026. web.stanford.edu/...slp3
- ^Rishi Bommasani et al., "On the Opportunities and Risks of Foundation Models," 2021. arxiv.org/...2108.07258
- ^Moritz Hardt, Eric Price, and Nati Srebro, "Equality of Opportunity in Supervised Learning," Advances in Neural Information Processing Systems 29, 2016. proceedings.neurips.cc/...7384ba499518b38-Abstract
- ^National Institute of Standards and Technology, "Towards a Standard for Identifying and Managing Bias in Artificial Intelligence," NIST Special Publication 1270, March 2022. doi.org/...NIST.SP.1270
- ^Ian Goodfellow et al., "Generative Adversarial Nets," Advances in Neural Information Processing Systems 27, 2014. proceedings.neurips.cc/...9a61f95710dbe25-Abstract
- ^Diederik P. Kingma and Max Welling, "Auto-Encoding Variational Bayes," 2013. arxiv.org/...1312.6114
- ^Jonathan Ho, Ajay Jain, and Pieter Abbeel, "Denoising Diffusion Probabilistic Models," Advances in Neural Information Processing Systems 33, 2020. proceedings.neurips.cc/...67f1ab10179ca4b-Abstract
- ^Ziwei Ji et al., "Survey of Hallucination in Natural Language Generation," ACM Computing Surveys 55(12), article 248, 2023. doi.org/...3571730
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.
7 revisions · v8 · 5,692 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independently verified against 19 primary, peer-reviewed, official, and author-maintained sources covering terminology scope, statistical learning, deep learning, evaluation, calibration, leakage, reinforcement learning, transformers, language processing, foundation models, fairness, bias, generative models, and hallucination; all 139 definitions, 203 citation calls, bibliographic metadata, source boundaries, and current documentation checked through 2026-07-28.
Cite this page: AI Wiki. "Machine learning terms." aiwiki.ai, updated 28 Jul 2026, fact-checked 28 Jul 2026. CC BY 4.0. https://aiwiki.ai/wiki/machine_learning_terms