Showing 1-60 of 314 articles
2024 Nobel Prizes in AI
In 2024, five artificial intelligence researchers won Nobel Prizes across two categories: the Nobel Prize in Physics went to John J. Hopfield and Geoffrey...
AI HistoryArtificial Intelligence
AI accelerator
An AI accelerator (also called an AI chip or neural processing unit) is a class of specialized hardware designed to run artificial intelligence workloads,...
AI HardwareAI Infrastructure
AI weather forecasting
AI weather forecasting is the use of machine learning and deep learning models, trained on decades of historical atmospheric data, to predict the weather...
AI for ScienceArtificial Intelligence
ALBERT
ALBERT (A Lite BERT) is a parameter-efficient variant of the BERT language model developed by researchers at Google Research and the Toyota Technological...
Natural Language ProcessingTransformer Models
AWQ (Activation-aware Weight Quantization)
Activation-aware Weight Quantization (AWQ) is a post-training quantization method for large language models that compresses weights to 4-bit (and optionally...
AI InferenceLarge Language Models
Aaron Courville
Aaron Courville is a Canadian computer scientist, a full professor in the Department of Computer Science and Operations Research (DIRO) at the Universite de...
People
Absolute Zero Reasoner
Absolute Zero is a reinforcement learning paradigm for training reasoning models in which a single model proposes its own tasks and then solves them, using...
Machine Learning
Activation Function
An activation function is a nonlinear mathematical function applied to the output of each neuron in an artificial neural network, and it is what gives the...
Machine LearningNeural Networks
Adafactor
Adafactor is a memory-efficient adaptive learning-rate optimizer for training deep neural networks, introduced by Noam Shazeer and Mitchell Stern in the 2018...
Training & Optimization
AlexNet
AlexNet is a deep learning convolutional neural network, built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton at the University of Toronto, that won...
Computer VisionMachine Learning
Andrej Karpathy
Andrej Karpathy (born October 23, 1986) is a Slovak-Canadian computer scientist, AI researcher, educator, and entrepreneur who was a founding member of OpenAI,...
AI CompaniesOpenAI
Attention
See also: Machine learning terms > This article gives a high-level overview of attention as a family of mechanisms in machine learning. For deeper treatments,...
Machine LearningNeural Networks
Attention Is All You Need
"Attention Is All You Need" is the 2017 research paper that introduced the transformer, the neural network architecture that underpins virtually every modern...
Machine LearningNatural Language Processing
Attention sink
An attention sink is an empirical phenomenon in Transformer language models in which a large fraction of each attention head's weight concentrates on a few...
Neural Networks
Audio Classification Models
Audio classification models are machine learning systems for audio classification, the task of assigning one or more labels to an audio recording or to short...
Machine LearningSpeech & Audio AI
AudioCraft
See also: Generative AI, Meta AI, and Deep Learning AudioCraft is an open-source generative-audio library released by Meta AI (Fundamental AI Research, FAIR)...
Generative AIMeta AI
Autoencoder
An autoencoder is a type of neural network trained to reconstruct its own input through a low-dimensional bottleneck representation, using the input itself as...
Generative AI
Backpropagation
Backpropagation (short for "backward propagation of errors") is the algorithm used to compute the gradient of a scalar loss function with respect to every...
Machine LearningNeural Networks
Bahdanau attention
Bahdanau attention is the first attention mechanism for neural networks, introduced in 2014 to let a sequence-to-sequence decoder soft-align to every encoder...
Model ArchitectureNatural Language Processing
Batch
See also: machine learning terms, batch size, gradient descent A batch in machine learning is the set of training examples processed together in one forward...
Machine Learning
Batch Normalization
See also: Machine learning terms Batch normalization (often abbreviated BatchNorm or BN) is a technique for improving the speed, stability, and performance of...
Machine LearningNeural Networks
Batch Size
In machine learning, batch size is the hyperparameter that sets how many training examples a model processes together before it updates its parameters with one...
Machine Learning
Bayesian Neural Network
A Bayesian neural network (BNN) is a neural network in which the weights and biases are represented as probability distributions rather than fixed point...
Machine LearningNeural Networks
Broadcasting
Broadcasting is the set of rules that lets element-wise operations (addition, subtraction, multiplication, division) act on arrays or tensors of different but...
Machine LearningMathematics
CLIP (Contrastive Language-Image Pre-training)
CLIP (Contrastive Language-Image Pre-training) is a multimodal neural network developed by OpenAI that learns visual concepts from natural language by training...
Computer VisionMachine Learning
Calibration Layer
A calibration layer is a post-prediction adjustment appended to a trained machine learning model that rescales its raw output scores or predicted probabilities...
Machine LearningModel Evaluation
Causal Language Model
A causal language model (CLM), also called an autoregressive language model or a decoder-only language model, is a language model that predicts the next token...
Machine LearningNatural Language Processing
Chain of Thought Monitorability
Chain of thought monitorability is the property that lets safety researchers read a reasoning model's chain-of-thought (CoT), the step-by-step working it...
Machine Learning
Chain-of-Thought
Chain-of-thought (CoT) prompting is a prompt engineering technique that improves the reasoning ability of large language models by having them generate a...
Machine LearningNatural Language Processing
Checkpoint
See also: Machine learning terms In machine learning, a checkpoint is a saved snapshot of a model's state captured at a specific point during the training...
Machine Learning
Chinchilla scaling laws
The Chinchilla scaling laws are a set of empirical findings published by DeepMind researchers in 2022 showing that, for a fixed compute budget, a large...
AI ResearchLarge Language Models
Classifier-Free Guidance (CFG)
Classifier-Free Guidance (CFG) is an inference-time technique that steers conditional generative models, especially diffusion models, by combining a single...
Generative AI
Clipping
Clipping is a family of techniques in machine learning that constrain numerical values to lie within a specified range or below a specified magnitude. The most...
Machine LearningTraining & Optimization
Co-Adaptation
Co-adaptation in neural networks refers to a phenomenon in which different hidden units develop highly correlated behavior, becoming excessively dependent on...
Machine LearningNeural Networks
Computational graph
A computational graph is a directed acyclic graph (DAG) representation of a numerical computation, where nodes represent operations (or variables) and edges...
Developer Tools
Computer vision
Computer vision is the field of artificial intelligence that enables computers to extract meaning from digital images, video, and 3D data, performing tasks...
Artificial IntelligenceComputer Vision
Context window
A context window (also called context length) is the maximum number of tokens that a large language model (LLM) can process at once, spanning both the input...
Artificial IntelligenceLarge Language Models
Continual learning
Continual learning, also called lifelong learning or incremental learning, is a machine learning paradigm in which a model learns from a stream of tasks or...
Machine LearningNeural Networks
Contrastive Learning
See also: self-supervised learning, representation learning, metric learning, transfer learning, deep learning Contrastive learning is a family of machine...
Machine Learning
ControlNet
ControlNet is a neural network architecture that adds spatial and structural control to large pretrained text-to-image diffusion models. It was introduced in a...
Computer VisionGenerative AI
ConvNeXt
ConvNeXt is a family of pure convolutional neural network (CNN) models that match or beat Vision Transformers on standard vision benchmarks, reaching 87.8%...
Computer VisionNeural Networks
Convolution
See also: Machine learning terms, Convolutional layer, Convolutional filter Convolution is a mathematical operation that combines two functions to produce a...
Machine LearningMathematics
Convolutional Filter
A convolutional filter (also called a kernel or feature detector) is a small matrix of learnable weights that slides across an input and computes a dot product...
Computer VisionMachine Learning
Convolutional Layer
See also: Machine learning terms A convolutional layer is the core building block of a convolutional neural network (CNN): it slides a small set of learnable...
Computer VisionMachine Learning
Convolutional Neural Network
A convolutional neural network (CNN or ConvNet) is a type of neural network that processes grid-like data such as images by sliding small learnable filters...
Computer VisionMachine Learning
Cosine learning rate schedule
The cosine learning rate schedule, also called cosine annealing, is a learning rate decay strategy that lowers the optimizer step size from a peak value to a...
Training & Optimization
Critic
A critic in reinforcement learning (RL) is the component of an actor-critic system that estimates a value function, scoring how good the actor's chosen actions...
Machine LearningReinforcement Learning
Cross-Entropy
See also: Machine learning terms, Loss function, Entropy Cross-entropy is a measure from information theory of how many bits (or nats) are needed to encode...
Machine LearningMathematics
Cross-Entropy Loss
Cross-entropy loss is the standard loss function for classification and language modeling, defined as the negative log-probability a model assigns to the...
Machine Learning
Curriculum learning
Curriculum learning is a training strategy for machine learning models in which training examples are presented in a meaningful, easy-to-hard order rather than...
Machine LearningTraining & Optimization
DCGAN (Deep Convolutional GAN)
DCGAN (Deep Convolutional Generative Adversarial Network) is a family of generative adversarial network architectures, introduced in 2015 by Alec Radford, Luke...
Computer VisionGenerative AI
DDPG (Deep Deterministic Policy Gradient)
DDPG (Deep Deterministic Policy Gradient) is an off-policy, model-free actor-critic algorithm in deep reinforcement learning that learns continuous-control...
Reinforcement Learning
DDPM
Denoising Diffusion Probabilistic Models (DDPM) are a class of generative model introduced by Jonathan Ho, Ajay Jain, and Pieter Abbeel of UC Berkeley in their...
Generative AIMachine Learning
DETR
DETR (DEtection TRansformer) is an end-to-end object detection model that reframes detection as a direct set prediction problem solved with a transformer...
Computer VisionTransformer Models
DINO (computer vision)
DINO (self-DIstillation with NO labels) is a family of self-supervised learning methods for computer vision from Meta AI that trains Vision Transformers (ViTs)...
Computer VisionMachine Learning
DPM-Solver
DPM-Solver is a fast, training-free high-order numerical solver for the ordinary differential equations (ODEs) that arise when sampling from diffusion models,...
Generative AI
DQN
The Deep Q-Network (DQN) is a model-free, off-policy reinforcement learning algorithm that combines Q-learning with a deep neural network function...
Google DeepMindReinforcement Learning
Data Augmentation
Data augmentation is a set of techniques that artificially expand the size and diversity of a training dataset by applying label-preserving transformations to...
Data & DatasetsMachine Learning
Data Parallelism
Data parallelism is a distributed training technique in which the same neural network model is replicated across multiple processing units (typically GPUs),...
AI InfrastructureMachine Learning
Dataset API (tf.data)
The Dataset API (tf.data) is the high-performance input pipeline framework within TensorFlow for loading, transforming, and delivering data to machine learning...
Developer ToolsMachine Learning