Deep Learning

Explore Deep Learning through related topics and the articles other pages reference most.

333 articlesSearch and filter Follow updates by RSS

Explore articles

Browse subtopics: Neural Networks

Articles that also belong to these categories. Counts cover all of Deep Learning.

All articles (333)Machine Learning (187)Neural Networks (81)Computer Vision (53)Training & Optimization (41)Natural Language Processing (37)Generative AI (31)Model Architecture (21)Transformer Models (18)Artificial Intelligence (16)Large Language Models (12)People (12)Reinforcement Learning (12)Developer Tools (11)Mathematics (11)AI Inference (9)Interpretability (8)AI Infrastructure (7)Speech & Audio AI (7)AI Hardware (6)AI Companies (5)AI History (5)Google DeepMind (5)Open Source AI (5)AI Research (4)AI for Science (4)Algorithms (4)Model Evaluation (4)Robotics (4)AI Models (3)Google (3)Meta AI (3)OpenAI (3)AI Alignment (2)AI Tools & Products (2)Chinese AI (2)Data & Datasets (2)Healthcare AI (2)MLOps (2)Microsoft (2)Multimodal AI (2)NVIDIA (2)Software Development (2)Statistics (2)AI Benchmarks (1)AI Ethics (1)AI Safety (1)Data Science (1)Diffusion Models (1)Embodied AI (1)Image Generation (1)Information Retrieval (1)Programming Languages (1)Prompt Engineering (1)Video Generation (1)

Showing 61-81 of 81 articles

RWKV

RWKV (pronounced "RwaKuv") is an open-source neural network architecture that combines the parallelizable training of Transformers with the constant-time

Machine Learning Model Architecture

ReLU

ReLU, short for rectified linear unit, is an activation function that maps a real input to its positive part:

Machine Learning Neural Networks

The Rectified Linear Unit (ReLU) is the most widely used activation function in deep learning, defined mathematically as $$f(x) = \max(0, x)$$: it returns the input directly when positive and outputs zero…

Machine Learning Neural Networks

Recurrent Neural Network

A recurrent neural network (RNN) is a neural network whose computation includes a state that is passed from one step to the next.

Machine Learning Model Architecture

Register tokens (Vision Transformers Need Registers)

Register tokens are a small set of extra learnable tokens added to the input sequence of a Vision Transformer (ViT) so the network has a dedicated place to carry out internal, image-level computation.

Neural Networks

ResNet

ResNet, short for residual network, is a family of deep convolutional neural networks introduced by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun at CVPR 2016.

Computer Vision Neural Networks

Self-Extend

Self-Extend (written SelfExtend in the original paper) is a training-free technique that lets a pretrained large language model process inputs much longer than the context window it was trained on, with no…

Neural Networks

Self-attention

Self-attention is a neural network operation in which each position forms a data-dependent mixture of information from positions in the same input sequence or set.

Machine Learning Model Architecture

Sigmoid Function

The sigmoid function is a mathematical function that maps any real number to a value between 0 and 1 using the formula $$\sigma(x) = \frac{1}{1 + e^{-x}}$$, producing a smooth S-shaped curve.

Machine Learning Mathematics

Soft MoE

Soft MoE (Soft Mixture of Experts) is a fully differentiable variant of the sparse mixture of experts (MoE) layer.

Neural Networks

Softmax

Softmax is a function that converts a finite vector of real-valued scores into a vector of positive numbers that sum to one.

Machine Learning Neural Networks

SwiGLU

SwiGLU (Swish-Gated Linear Unit) is the activation function used inside the feed-forward sublayer of most modern transformer large language models, including LLaMA, PaLM, Mistral, Qwen, and DeepSeek.

Model Architecture Neural Networks

Swin Transformer

The Swin Transformer (Shifted Window Transformer) is a hierarchical vision transformer architecture that computes self-attention within local, non-overlapping windows and introduces a shifted window…

Computer Vision Neural Networks

Tanh (hyperbolic tangent)

The hyperbolic tangent, written tanh, is a smooth, S-shaped activation function that maps any real number into the open interval $$(-1, 1)$$, passing through the origin so that tanh(0) = 0 .

Neural Networks

Transformers

A Transformer is a deep learning architecture, introduced by eight Google researchers in the 2017 paper "Attention Is All You Need", that uses attention as the sole mechanism for modeling relationships between…

Model Architecture Neural Networks

Translational invariance

Translational invariance (also called translation invariance or shift invariance) is the property of a function, system, or machine learning model whose output does not change when its input is translated…

Computer Vision Neural Networks

VGG

VGG (also called VGGNet) is a deep convolutional neural network architecture, introduced in 2014 by Karen Simonyan and Andrew Zisserman of the Visual Geometry Group at the University of Oxford

Computer Vision Neural Networks

Vanishing Gradient Problem

The vanishing gradient problem is a difficulty in training deep neural networks where the gradients used to update the network shrink exponentially as they are propagated backward through the layers, leaving…

Machine Learning Neural Networks

YOLO (object detection)

YOLO (You Only Look Once) is a family of object detection models that treat detection as a single regression problem, predicting bounding boxes and class probabilities directly from full images in one forward…

Computer Vision Neural Networks

tf.keras

tf.keras is the high-level deep learning API built directly into the TensorFlow machine learning framework

Developer Tools Machine Learning

xLSTM

xLSTM (Extended Long Short-Term Memory) is a recurrent neural network architecture introduced in May 2024 by Maximilian Beck, Korbinian Pöppel, Sepp Hochreiter, and collaborators at Johannes Kepler University…

Model Architecture Neural Networks