Estimator (tf.estimator)
tf.estimator is a high-level TensorFlow API that encapsulates the complete lifecycle of a machine learning model, including training, evaluation, prediction, and export for serving.
Deep Learning
Explore Deep Learning through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Deep Learning.
All articles (333)Machine Learning (187)Neural Networks (81)Computer Vision (53)Training & Optimization (41)Natural Language Processing (37)Generative AI (31)Model Architecture (21)Transformer Models (18)Artificial Intelligence (16)Large Language Models (12)People (12)Reinforcement Learning (12)Developer Tools (11)Mathematics (11)AI Inference (9)Interpretability (8)AI Infrastructure (7)Speech & Audio AI (7)AI Hardware (6)AI Companies (5)AI History (5)Google DeepMind (5)Open Source AI (5)AI Research (4)AI for Science (4)Algorithms (4)Model Evaluation (4)Robotics (4)AI Models (3)Google (3)Meta AI (3)OpenAI (3)AI Alignment (2)AI Tools & Products (2)Chinese AI (2)Data & Datasets (2)Healthcare AI (2)MLOps (2)Microsoft (2)Multimodal AI (2)NVIDIA (2)Software Development (2)Statistics (2)AI Benchmarks (1)AI Ethics (1)AI Safety (1)Data Science (1)Diffusion Models (1)Embodied AI (1)Image Generation (1)Information Retrieval (1)Programming Languages (1)Prompt Engineering (1)Video Generation (1)
Showing 61-120 of 187 articles
Exploding Gradient Problem
The exploding gradient problem is a training failure in deep neural networks where the gradients of the loss function grow exponentially large as they propagate backward through layers during backpropagation…
Federated Learning
Federated learning is a machine learning technique that trains a shared model across many decentralized devices or servers without moving their raw data to a central location.
Feedforward Neural Network (FFN)
A feedforward neural network (FFN), also called a multilayer perceptron (MLP) when it has multiple layers, is a type of artificial neural network in which information flows in one direction only, from the…
Few-Shot Learning
Few-shot learning is a branch of machine learning in which a model learns to recognize new classes or perform new tasks from only a very small number of labeled examples, typically one to five per class .
Fine Tuning
Fine-tuning is the process of adapting a pretrained machine-learning model by continuing to optimize some or all of its parameters on data selected for a target task, domain, behavior, or population.
Focal loss
Focal loss is a loss function that reshapes standard cross-entropy loss by adding a (1 - p_t)^gamma modulating factor, which down-weights well-classified (easy) examples so that training concentrates on hard
Full Softmax
Full softmax (also called the standard softmax or exact softmax) is the softmax computation that calculates a probability for every possible output class in a classification model
Fully Connected Layer
A fully connected layer (also called a dense layer or linear layer) is a layer in an artificial neural network in which every input value connects to every output value through a learned weight
GPU computing
GPU computing is the use of a graphics processing unit (GPU) to perform general-purpose computation that was traditionally handled by the central processing unit (CPU).
Generalization
Generalization in machine learning is the ability of a trained model to perform accurately on new, unseen data drawn from the same distribution as its training set
Generative Model
A generative model is a class of statistical and machine learning model that learns the joint probability distribution P(X) of the observed data, or the joint distribution P(X, Y) of inputs and labels
Generative adversarial network
Generative adversarial networks (GANs) are a family of generative models trained through competition between two learned functions.
Geometric Deep Learning
Geometric deep learning (GDL) is an area of deep learning that designs neural networks around the structure and symmetries of their data.
Grad-CAM
Grad-CAM (Gradient-weighted Class Activation Mapping) is a technique for producing visual explanations from convolutional neural network (CNN) models by using the gradients of a target class flowing into the…
Gradient Accumulation
Gradient accumulation is a deep learning training technique that simulates a large batch size on limited GPU memory by summing the gradients from several small mini-batches (called micro-batches) and…
Gradient Descent
Gradient descent is an iterative first-order method for minimizing a differentiable scalar objective. Starting from a point, it evaluates the local gradient and moves in the opposite direction.
Graph Execution
Graph execution is a computation paradigm in machine learning frameworks where mathematical operations are organized into a directed acyclic graph (DAG) before being executed, so the runtime sees the whole…
Grokking
Grokking, also called delayed generalization, is a phenomenon in deep learning where a neural network first memorizes its training data (achieving near-perfect training accuracy but random-level test…
Grouped-Query Attention
Grouped-query attention (GQA) is an attention mechanism for transformer language models that partitions the query heads into a small number of groups, where every query head in a group shares one key…
Hidden Layer
A hidden layer is a layer of artificial neurons in a neural network that sits between the input layer and the output layer
Hopfield Network
A Hopfield network is a form of recurrent neural network that stores information as stable patterns in a system of interconnected binary units and retrieves it by content rather than by address.
Hyperparameter
A hyperparameter is an input that configures a machine learning algorithm or modeling pipeline, rather than an output fitted by that algorithm in one training run.
Image Recognition
Image recognition is the field of artificial intelligence and computer vision that enables machines to identify, classify, and interpret the objects, patterns, and features contained in a digital image or…
Image classification
Image classification is the task of assigning a whole image to one category drawn from a fixed set of labels.
Image segmentation
Image segmentation is the computer vision task of partitioning a digital image into multiple regions by assigning every pixel a label, producing a pixel-level map of what each part of the image contains.
ImageNet
ImageNet is a large, hierarchically organized image dataset created to support research in computer vision, especially object recognition.
In-Context Learning
In-context learning (ICL) is the ability of a large language model to learn a new task at inference time by conditioning on a prompt that contains a few input-output examples (demonstrations)
Inductive bias
Inductive bias (also called learning bias) is the set of assumptions that a learning algorithm uses to predict outputs for previously unseen inputs.
Inference
In machine learning, inference is the execution of a trained model to produce an output from an input.
Input Layer
The input layer is the first layer of a neural network: it receives the raw feature vector for each data sample and passes those values forward to the next layer, performing no learned computation of its own.
Instruction Tuning
Instruction tuning is the post-pretraining training stage in which a large language model (LLM) is fine-tuned on a curated collection of (instruction
Integrated Gradients
Integrated Gradients (IG) is a feature-attribution method for explainable AI that explains a neural network prediction by assigning each input feature an importance score
Iteration
An iteration in machine learning is a single update of a model's parameters during training, performed by processing one batch of data: the model makes predictions on the batch (a forward pass), computes the…
JAX
JAX is an open-source Python library for accelerator-oriented array computation and program transformation.
KV Cache
KV cache, short for key-value cache, is transient model state used during Transformer generation.
Keras
Keras is an open-source, high-level neural network API written in Python that lets developers build, train, and deploy deep learning models with minimal code.
Knowledge Distillation
Knowledge distillation is a training method in which a student model learns from signals produced by a teacher model.
Kolmogorov-Arnold Network
A Kolmogorov-Arnold Network (KAN) is a type of neural network architecture proposed as an alternative to the traditional Multi-Layer Perceptron (MLP).
Language Model
A language model is a model of patterns in language that assigns probabilities or comparable scores to linguistic sequences. Its units may be characters, words, subwords, bytes, or other tokens.
Large Language Model
A large language model (LLM) is an artificial intelligence system built on a transformer neural network with billions to trillions of parameters, trained on massive text corpora to predict the next token in a…
Latent Space
A latent space is the vector space a machine learning model maps its inputs into, where each input becomes a point (a latent vector or latent code) and the geometry of the space carries information the raw…
Latent reasoning via recurrent depth (Huginn)
Latent reasoning via recurrent depth is an approach to scaling a language model's test-time computation by iterating a recurrent transformer block in latent (hidden) space
Layer
A layer is the fundamental building block of a neural network: an organized group of neurons (also called nodes or units) that together apply one mathematical transformation to their input and pass the result…
Layer-wise Relevance Propagation (LRP)
Layer-wise Relevance Propagation (LRP) is an explainable AI method that explains the prediction of a deep neural network by propagating the model's output backward through the network, layer by layer, and…
Learning Rate
The learning rate is a hyperparameter that scales an update made by an iterative optimization algorithm. In machine learning, it is commonly written as η or α.
LoRA (Low-Rank Adaptation)
Low-Rank Adaptation, usually abbreviated LoRA, is a parameter-efficient fine-tuning method for adapting a pre-trained model.
Logits
In machine learning and statistics, logits refer to the raw, unnormalized scores output by a model before they are converted into probabilities.
Loss Curve
A loss curve is a plot that shows the value of a loss function over the course of training a machine learning model.
Loss Function
A loss function assigns a numerical cost to a prediction, decision, or action after an outcome is observed.
Loss Surface
The loss surface (also called the loss landscape, the error surface, or the objective function surface) is the geometric representation of a loss function as a function of the model's parameters.
Machine learning terms/TensorFlow
TensorFlow is an open-source software library for machine learning, deep learning, and numerical computation, developed and maintained by Google and first released to the public under the Apache 2.0 license on…
Manifold Hypothesis
The manifold hypothesis is the conjecture that real-world high-dimensional data, such as natural images, speech, and text representations, concentrates on or near a manifold of much lower dimension embedded…
Masked Language Model
A masked language model (MLM) is a language model trained to predict missing tokens that have been hidden in a sequence of text, using context from both the left and the right of each blank.
Meta-Learning
Meta-learning, often called "learning to learn", is a branch of machine learning in which a model is trained across many different tasks so that it can master a new task from only a handful of examples or a…
Mixed-Precision Training
Mixed-precision training is a technique for training deep learning models using lower-precision floating-point formats for most computations while maintaining a higher-precision copy of the model weights for…
Mixture of Depths
Mixture of Depths (MoD) is a technique for dynamically allocating computation to individual tokens within transformer-based language models.
Mixture of Experts (MoE)
A mixture of experts (MoE) is a machine-learning architecture that contains multiple component functions called experts and a gating or routing mechanism that determines how their outputs contribute to a…
Modality
A modality in machine learning and artificial intelligence is a distinct type, form, or structure of data that a model can process, learn from, or generate.
Model Parallelism
Model parallelism is a distributed training and inference technique that splits a single neural network across multiple processing units so that no individual accelerator has to hold the entire model.