2024 Nobel Prizes in AI
In 2024, five artificial intelligence researchers won Nobel Prizes across two categories: the Nobel Prize in Physics went to John J. Hopfield and Geoffrey Hinton "for foundational discoveries and inventions…
Deep Learning
Explore Deep Learning through related topics and the articles other pages reference most.
Most referenced in this topic
Ranked by links from other AI Wiki pages.
Explore articles
Browse subtopics (54)
Articles that also belong to these categories. Counts cover all of Deep Learning.
All articles (333)Machine Learning (187)Neural Networks (81)Computer Vision (53)Training & Optimization (41)Natural Language Processing (37)Generative AI (31)Model Architecture (21)Transformer Models (18)Artificial Intelligence (16)Large Language Models (12)People (12)Reinforcement Learning (12)Developer Tools (11)Mathematics (11)AI Inference (9)Interpretability (8)AI Infrastructure (7)Speech & Audio AI (7)AI Hardware (6)AI Companies (5)AI History (5)Google DeepMind (5)Open Source AI (5)AI Research (4)AI for Science (4)Algorithms (4)Model Evaluation (4)Robotics (4)AI Models (3)Google (3)Meta AI (3)OpenAI (3)AI Alignment (2)AI Tools & Products (2)Chinese AI (2)Data & Datasets (2)Healthcare AI (2)MLOps (2)Microsoft (2)Multimodal AI (2)NVIDIA (2)Software Development (2)Statistics (2)AI Benchmarks (1)AI Ethics (1)AI Safety (1)Data Science (1)Diffusion Models (1)Embodied AI (1)Image Generation (1)Information Retrieval (1)Programming Languages (1)Prompt Engineering (1)Video Generation (1)
Showing 1-60 of 333 articles
AI weather forecasting
AI weather forecasting is the use of machine learning and deep learning models, trained on decades of historical atmospheric data, to predict the weather faster and often more accurately than the physics-based…
ALBERT
ALBERT (A Lite BERT) is a parameter-efficient variant of the BERT language model developed by researchers at Google Research and the Toyota Technological Institute at Chicago (TTIC).
AWQ (Activation-aware Weight Quantization)
Activation-aware Weight Quantization (AWQ) is a post-training quantization method for large language models that compresses weights to 4-bit (and optionally 3-bit) integers while keeping near-FP16 task…
Aaron Courville
Aaron Courville is a Canadian computer scientist, a full professor in the Department of Computer Science and Operations Research (DIRO) at the Universite de Montreal, and the scientific director of IVADO, best…
Absolute Zero Reasoner
Absolute Zero Reasoner (AZR) is a research system for post-training a pretrained language model through self-play on machine-generated Python reasoning tasks.
Activation Function
An activation function is a nonlinear mathematical function applied to the output of each neuron in an artificial neural network, and it is what gives the network the ability to learn complex
Actor-critic methods
Actor-critic methods are a family of reinforcement learning algorithms that learn two things at once: a parameterized policy, called the actor, which chooses actions, and a value function, called the critic
Adafactor
Adafactor is an adaptive optimizer designed to reduce the memory used by second-moment estimates during neural-network training.
AlexNet
AlexNet is a deep learning convolutional neural network, built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton at the University of Toronto, that won the ImageNet Large Scale Visual Recognition…
Andrej Karpathy
Andrej Karpathy (born October 23, 1986) is a Slovak-Canadian computer scientist, AI researcher, educator, and entrepreneur who was a founding member of OpenAI, served as Tesla's Senior Director of AI and head…
Attention
Attention is a family of operations in neural networks that computes an output by assigning data-dependent weights to a collection of representations and combining them.
Attention Is All You Need
"Attention Is All You Need" is the 2017 research paper that introduced the transformer, the neural network architecture that underpins virtually every modern large language model.
Attention sink
An attention sink is an empirical phenomenon in Transformer language models in which a large fraction of each attention head's weight concentrates on a few tokens at the very start of the sequence, most often…
Audio Classification Models
Audio classification models are machine learning systems for audio classification, the task of assigning one or more labels to an audio recording or to short fragments of it.
AudioCraft
AudioCraft is an open-source generative-audio library released by Meta AI (Fundamental AI Research, FAIR) on August 2, 2023 that generates high-quality music and sound from text prompts using a single…
Autoencoder
An autoencoder is a type of neural network trained to reconstruct its own input through a low-dimensional bottleneck representation, using the input itself as the training signal rather than any external label.
Backpropagation
Backpropagation is an algorithm for computing derivatives of a scalar objective with respect to variables in a layered or otherwise composed computation.
Bahdanau attention
Bahdanau attention is the first attention mechanism for neural networks, introduced in 2014 to let a sequence-to-sequence decoder soft-align to every encoder hidden state instead of relying on a single…
Batch
A batch in machine learning is the set of training examples processed together in one forward and backward pass before the model's parameters are updated once.
Batch Normalization
Batch normalization (BatchNorm or BN) is a normalization method used inside some neural networks.
Batch Size
In machine learning, batch size is the hyperparameter that sets how many training examples a model processes together before it updates its parameters with one step of gradient descent.
Bayesian Neural Network
A Bayesian neural network (BNN) is a neural network in which the weights and biases are represented as probability distributions rather than fixed point estimates.
Brain-Computer Interface
A brain-computer interface (BCI), also called a brain-machine interface, is a system that measures activity in the nervous system and translates it into commands for an external device, bypassing muscles and…
Brain2Qwerty
Brain2Qwerty is a research program in non-invasive brain-computer interfaces for decoding typed language from recordings of brain activity.
Broadcasting
Broadcasting is the set of rules that lets element-wise operations (addition, subtraction, multiplication, division) act on arrays or tensors of different but compatible shapes by virtually stretching the…
CLIP (Contrastive Language-Image Pre-training)
CLIP, short for Contrastive Language-Image Pre-training, is a family of neural networks developed by OpenAI researchers to learn a shared representation of images and natural-language descriptions.
Calibration Layer
A calibration layer is a post-prediction adjustment appended to a trained machine learning model that rescales its raw output scores or predicted probabilities so they better reflect the true likelihood of…
Causal Language Model
A causal language model (CLM), also called an autoregressive language model or a decoder-only language model, is a language model that predicts the next token in a sequence using only the tokens that precede…
Chain of Thought Monitorability
Chain of thought monitorability is the property that lets safety researchers read a reasoning model's chain-of-thought (CoT), the step-by-step working it writes out in human language before answering, and…
Chain-of-Thought
Chain-of-thought (CoT) is a sequence of intermediate steps that a language model generates before its final answer.
Checkpoint
In machine learning, a checkpoint is a saved snapshot of a model's state captured at a specific point during the training process, used to resume training, recover from failures, or deploy and fine-tune the…
Chinchilla scaling laws
The Chinchilla scaling laws are a set of empirical findings published by DeepMind researchers in 2022 showing that, for a fixed compute budget, a large language model trains most efficiently when its number of…
Classifier-Free Guidance (CFG)
Classifier-Free Guidance (CFG) is an inference-time technique that steers conditional generative models, especially diffusion models, by combining a single model's conditional and unconditional predictions and…
Clipping
Clipping is a family of techniques in machine learning that constrain numerical values to lie within a specified range or below a specified magnitude.
Co-Adaptation
Co-adaptation in neural networks refers to a phenomenon in which different hidden units develop highly correlated behavior, becoming excessively dependent on one another rather than learning independent
Computational graph
A computational graph is a directed acyclic graph (DAG) representation of a numerical computation, where nodes represent operations (or variables) and edges represent the data, typically tensors
Computer Vision
Computer vision is the study of computational methods that extract, estimate, or generate useful representations from visual measurements.
Connectionism
Connectionism is an approach in cognitive science and artificial intelligence that explains mental phenomena as the collective behavior of large networks of simple, interconnected processing units.
Connectionist Temporal Classification
Connectionist temporal classification (CTC) is a loss function and output layer design for training neural networks to label unsegmented sequences, such as transcribing an audio recording into characters when…
Context window
A context window is the finite token sequence that a language model can process for one invocation.
Continual learning
Continual learning, also called lifelong learning or incremental learning, is a machine learning paradigm in which a model learns from a stream of tasks or data distributions over time
Contrastive Learning
Contrastive learning is a family of machine learning methods that learn representations by pulling similar (positive) pairs of data points closer together in an embedding space while pushing dissimilar…
ControlNet
ControlNet is a neural network architecture that adds spatial and structural control to large pretrained text-to-image diffusion models.
ConvNeXt
ConvNeXt is a family of pure convolutional neural network (CNN) models that match or beat Vision Transformers on standard vision benchmarks
Convolution
Convolution is a mathematical operation that combines two functions to produce a third function, expressing how the shape of one is modified by the other.
Convolutional Filter
A convolutional filter (also called a kernel or feature detector) is a small matrix of learnable weights that slides across an input and computes a dot product at each position to produce a feature map.
Convolutional Layer
A convolutional layer is the core building block of a convolutional neural network (CNN): it slides a small set of learnable filters (also called kernels) across the input, computing a convolution (technically…
Convolutional Neural Network
A convolutional neural network (CNN, or ConvNet) is a neural network that uses convolution-like linear operators in at least some layers.
Cosine learning rate schedule
The cosine learning rate schedule, also called cosine annealing, is a learning rate decay strategy that lowers the optimizer step size from a peak value to a small minimum following a half period of a cosine…
Critic
A critic in reinforcement learning (RL) is the component of an actor-critic system that estimates a value function, scoring how good the actor's chosen actions are and converting that judgment into a…
Cross-Entropy
Cross-entropy is a measure from information theory of how many bits (or nats) are needed to encode data drawn from a true probability distribution P when using a code optimized for a different
Cross-Entropy Loss
Cross-entropy loss is the standard loss function for classification and language modeling, defined as the negative log-probability a model assigns to the correct answer: for a single example its value is…
Curriculum learning
Curriculum learning is a training strategy for machine learning models in which training examples are presented in a meaningful, easy-to-hard order rather than at random
DCGAN (Deep Convolutional GAN)
DCGAN (Deep Convolutional Generative Adversarial Network) is a family of generative adversarial network architectures, introduced in 2015 by Alec Radford, Luke Metz, and Soumith Chintala
DDPG (Deep Deterministic Policy Gradient)
DDPG (Deep Deterministic Policy Gradient) is an off-policy, model-free actor-critic algorithm in deep reinforcement learning that learns continuous-control policies by combining a deterministic actor with a…
DDPM
Denoising Diffusion Probabilistic Models (DDPM) are a class of generative model introduced by Jonathan Ho, Ajay Jain, and Pieter Abbeel of UC Berkeley in their June 2020 paper "Denoising Diffusion…
DETR
DETR (DEtection TRansformer) is an end-to-end object detection model that reframes detection as a direct set prediction problem solved with a transformer encoder-decoder and bipartite matching, removing the…
DINO (computer vision)
DINO (self-DIstillation with NO labels) is a family of self-supervised learning methods for computer vision from Meta AI that trains Vision Transformers (ViTs) on unlabeled images and produces general-purpose…
DPM-Solver
DPM-Solver is a fast, training-free high-order numerical solver for the ordinary differential equations (ODEs) that arise when sampling from diffusion models, and it lets a pretrained model generate…