Deep Learning
332 AI Wiki articles on Deep Learning. The most referenced are Large Language Model, Deep Learning and Transformers.
332 articlesRSS
Showing 1-60 of 332 articles
2024 Nobel Prizes in AI
In 2024, five artificial intelligence researchers won Nobel Prizes across two categories: the Nobel Prize in Physics went to John J. Hopfield and Geoffrey...
AI HistoryArtificial Intelligence
AI weather forecasting
AI weather forecasting is the use of machine learning and deep learning models, trained on decades of historical atmospheric data, to predict the weather...
AI for ScienceArtificial Intelligence
ALBERT
ALBERT (A Lite BERT) is a parameter-efficient variant of the BERT language model developed by researchers at Google Research and the Toyota Technological...
Natural Language ProcessingTransformer Models
AWQ (Activation-aware Weight Quantization)
Activation-aware Weight Quantization (AWQ) is a post-training quantization method for large language models that compresses weights to 4-bit (and optionally...
AI InferenceLarge Language Models
Aaron Courville
Aaron Courville is a Canadian computer scientist, a full professor in the Department of Computer Science and Operations Research (DIRO) at the Universite de...
People
Absolute Zero Reasoner
Absolute Zero Reasoner (AZR) is a research system for post-training a pretrained language model through self-play on machine-generated Python reasoning tasks....
Machine Learning
Activation Function
An activation function is a nonlinear mathematical function applied to the output of each neuron in an artificial neural network, and it is what gives the...
Machine LearningNeural Networks
Actor-critic methods
Actor-critic methods are a family of reinforcement learning algorithms that learn two things at once: a parameterized policy, called the actor, which chooses...
AlgorithmsMachine Learning
Adafactor
Adafactor is an adaptive optimizer designed to reduce the memory used by second-moment estimates during neural-network training. Noam Shazeer and Mitchell...
Training & Optimization
AlexNet
AlexNet is a deep learning convolutional neural network, built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton at the University of Toronto, that won...
Computer VisionMachine Learning
Andrej Karpathy
Andrej Karpathy (born October 23, 1986) is a Slovak-Canadian computer scientist, AI researcher, educator, and entrepreneur who was a founding member of OpenAI,...
AI CompaniesOpenAI
Attention
Attention is a family of operations in neural networks that computes an output by assigning data-dependent weights to a collection of representations and...
Machine LearningNeural Networks
Attention Is All You Need
"Attention Is All You Need" is the 2017 research paper that introduced the transformer, the neural network architecture that underpins virtually every modern...
Machine LearningNatural Language Processing
Attention sink
An attention sink is an empirical phenomenon in Transformer language models in which a large fraction of each attention head's weight concentrates on a few...
Neural Networks
Audio Classification Models
See also: Audio Models and Audio Audio classification models are machine learning systems for audio classification, the task of assigning one or more labels to...
Machine LearningSpeech & Audio AI
AudioCraft
See also: Generative AI, Meta AI, and Deep Learning AudioCraft is an open-source generative-audio library released by Meta AI (Fundamental AI Research, FAIR)...
Generative AIMeta AI
Autoencoder
An autoencoder is a type of neural network trained to reconstruct its own input through a low-dimensional bottleneck representation, using the input itself as...
Generative AI
Backpropagation
Backpropagation is an algorithm for computing derivatives of a scalar objective with respect to variables in a layered or otherwise composed computation. It...
Machine LearningNeural Networks
Bahdanau attention
Bahdanau attention is the first attention mechanism for neural networks, introduced in 2014 to let a sequence-to-sequence decoder soft-align to every encoder...
Model ArchitectureNatural Language Processing
Batch
See also: machine learning terms, batch size, gradient descent A batch in machine learning is the set of training examples processed together in one forward...
Machine Learning
Batch Normalization
Batch normalization (BatchNorm or BN) is a normalization method used inside some neural networks. During training, it standardizes each selected activation...
Machine LearningNeural Networks
Batch Size
In machine learning, batch size is the hyperparameter that sets how many training examples a model processes together before it updates its parameters with one...
Machine Learning
Bayesian Neural Network
See also: Machine learning terms A Bayesian neural network (BNN) is a neural network in which the weights and biases are represented as probability...
Machine LearningNeural Networks
Brain-Computer Interface
A brain-computer interface (BCI), also called a brain-machine interface, is a system that measures activity in the nervous system and translates it into...
AI CompaniesAI for Science
Brain2Qwerty
Brain2Qwerty is a research program in non-invasive brain-computer interfaces for decoding typed language from recordings of brain activity. Meta AI introduced...
AI ResearchHealthcare AI
Broadcasting
See also: Machine learning terms Broadcasting is the set of rules that lets element-wise operations (addition, subtraction, multiplication, division) act on...
Machine LearningMathematics
CLIP (Contrastive Language-Image Pre-training)
CLIP, short for Contrastive Language-Image Pre-training, is a family of neural networks developed by OpenAI researchers to learn a shared representation of...
Computer VisionMachine Learning
Calibration Layer
A calibration layer is a post-prediction adjustment appended to a trained machine learning model that rescales its raw output scores or predicted probabilities...
Machine LearningModel Evaluation
Causal Language Model
A causal language model (CLM), also called an autoregressive language model or a decoder-only language model, is a language model that predicts the next token...
Machine LearningNatural Language Processing
Chain of Thought Monitorability
Chain of thought monitorability is the property that lets safety researchers read a reasoning model's chain-of-thought (CoT), the step-by-step working it...
Machine Learning
Chain-of-Thought
Chain-of-thought (CoT) is a sequence of intermediate steps that a language model generates before its final answer. Chain-of-thought prompting is the narrower...
Machine LearningNatural Language Processing
Checkpoint
See also: Machine learning terms In machine learning, a checkpoint is a saved snapshot of a model's state captured at a specific point during the training...
Machine Learning
Chinchilla scaling laws
The Chinchilla scaling laws are a set of empirical findings published by DeepMind researchers in 2022 showing that, for a fixed compute budget, a large...
AI ResearchLarge Language Models
Classifier-Free Guidance (CFG)
Classifier-Free Guidance (CFG) is an inference-time technique that steers conditional generative models, especially diffusion models, by combining a single...
Generative AI
Clipping
Clipping is a family of techniques in machine learning that constrain numerical values to lie within a specified range or below a specified magnitude. The most...
Machine LearningTraining & Optimization
Co-Adaptation
Co-adaptation in neural networks refers to a phenomenon in which different hidden units develop highly correlated behavior, becoming excessively dependent on...
Machine LearningNeural Networks
Computational graph
A computational graph is a directed acyclic graph (DAG) representation of a numerical computation, where nodes represent operations (or variables) and edges...
Developer Tools
Computer Vision
Computer vision is the study of computational methods that extract, estimate, or generate useful representations from visual measurements. Inputs can include...
Artificial IntelligenceComputer Vision
Connectionism
Connectionism is an approach in cognitive science and artificial intelligence that explains mental phenomena as the collective behavior of large networks of...
AI HistoryArtificial Intelligence
Connectionist Temporal Classification
Connectionist temporal classification (CTC) is a loss function and output layer design for training neural networks to label unsegmented sequences, such as...
Machine LearningSpeech & Audio AI
Context window
A context window is the finite token sequence that a language model can process for one invocation. For a text-generating model, that sequence can include...
Artificial IntelligenceLarge Language Models
Continual learning
See also: Transfer learning, Meta-learning, Deep learning, Neural network Continual learning, also called lifelong learning or incremental learning, is a...
Machine LearningNeural Networks
Contrastive Learning
See also: self-supervised learning, representation learning, metric learning, transfer learning, deep learning Contrastive learning is a family of machine...
Machine Learning
ControlNet
ControlNet is a neural network architecture that adds spatial and structural control to large pretrained text-to-image diffusion models. It was introduced in a...
Computer VisionGenerative AI
ConvNeXt
ConvNeXt is a family of pure convolutional neural network (CNN) models that match or beat Vision Transformers on standard vision benchmarks, reaching 87.8%...
Computer VisionNeural Networks
Convolution
See also: Machine learning terms, Convolutional layer, Convolutional filter Convolution is a mathematical operation that combines two functions to produce a...
Machine LearningMathematics
Convolutional Filter
A convolutional filter (also called a kernel or feature detector) is a small matrix of learnable weights that slides across an input and computes a dot product...
Computer VisionMachine Learning
Convolutional Layer
See also: Machine learning terms A convolutional layer is the core building block of a convolutional neural network (CNN): it slides a small set of learnable...
Computer VisionMachine Learning
Convolutional Neural Network
A convolutional neural network (CNN, or ConvNet) is a neural network that uses convolution-like linear operators in at least some layers. CNNs are especially...
Computer VisionMachine Learning
Cosine learning rate schedule
The cosine learning rate schedule, also called cosine annealing, is a learning rate decay strategy that lowers the optimizer step size from a peak value to a...
Training & Optimization
Critic
A critic in reinforcement learning (RL) is the component of an actor-critic system that estimates a value function, scoring how good the actor's chosen actions...
Machine LearningReinforcement Learning
Cross-Entropy
See also: Machine learning terms, Loss function, Entropy Cross-entropy is a measure from information theory of how many bits (or nats) are needed to encode...
Machine LearningMathematics
Cross-Entropy Loss
Cross-entropy loss is the standard loss function for classification and language modeling, defined as the negative log-probability a model assigns to the...
Machine Learning
Curriculum learning
Curriculum learning is a training strategy for machine learning models in which training examples are presented in a meaningful, easy-to-hard order rather than...
Machine LearningTraining & Optimization
DCGAN (Deep Convolutional GAN)
DCGAN (Deep Convolutional Generative Adversarial Network) is a family of generative adversarial network architectures, introduced in 2015 by Alec Radford, Luke...
Computer VisionGenerative AI
DDPG (Deep Deterministic Policy Gradient)
DDPG (Deep Deterministic Policy Gradient) is an off-policy, model-free actor-critic algorithm in deep reinforcement learning that learns continuous-control...
Reinforcement Learning
DDPM
Denoising Diffusion Probabilistic Models (DDPM) are a class of generative model introduced by Jonathan Ho, Ajay Jain, and Pieter Abbeel of UC Berkeley in their...
Generative AIMachine Learning
DETR
DETR (DEtection TRansformer) is an end-to-end object detection model that reframes detection as a direct set prediction problem solved with a transformer...
Computer VisionTransformer Models
DINO (computer vision)
DINO (self-DIstillation with NO labels) is a family of self-supervised learning methods for computer vision from Meta AI that trains Vision Transformers (ViTs)...
Computer VisionMachine Learning
DPM-Solver
DPM-Solver is a fast, training-free high-order numerical solver for the ordinary differential equations (ODEs) that arise when sampling from diffusion models,...
Generative AI