Depthwise Separable CNN
A depthwise separable convolution is a factorized form of convolution that decomposes a standard convolutional operation into two sequential steps: a depthwise convolution and a pointwise convolution.
Model Architecture
Explore Model Architecture through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Model Architecture.
All articles (65)Deep Learning (21)Transformer Models (16)Neural Networks (15)Large Language Models (14)Machine Learning (13)AI Models (6)Microsoft (6)Computer Vision (5)AI Inference (4)Meta AI (4)AI Companies (3)AI Infrastructure (2)Chinese AI (2)Google (2)Mixture of Experts (2)Natural Language Processing (2)Open Source AI (2)AI History (1)Artificial Intelligence (1)Developer Tools (1)Diffusion Models (1)Google DeepMind (1)Image Generation (1)Information Retrieval (1)MLOps (1)Multimodal AI (1)NVIDIA (1)People (1)Training & Optimization (1)
Showing 1-13 of 13 articles
Graph Machine Learning Models
Graph machine learning models are neural networks designed to operate on data structured as graphs, where the input is a set of nodes connected by edges rather than a grid like an image or a sequence like text.
Hidden Markov Model
A Hidden Markov Model (HMM) is a statistical model of sequential data in which an unobserved (hidden) sequence of discrete states follows a Markov process
Joint Embedding Predictive Architecture
Joint Embedding Predictive Architecture (JEPA) is a family of self-supervised, non-generative neural network architectures proposed by Yann LeCun in his June 2022 position paper A Path Towards Autonomous…
Machine learning terms/Sequence Models
Sequence models are a class of machine learning systems designed to process inputs or produce outputs that have a meaningful order.
Mixture of Depths
Mixture of Depths (MoD) is a technique for dynamically allocating computation to individual tokens within transformer-based language models.
Multi-Head Self-Attention
Multi-head self-attention is the core sequence-mixing mechanism of the Transformer architecture: it runs several scaled dot-product attention operations ("heads") in parallel over different learned projections…
Multi-head Latent Attention
Multi-head Latent Attention (MLA) is an attention mechanism for transformer models that achieves a 93.3% reduction in key-value cache size while maintaining or exceeding the performance of traditional…
RWKV
RWKV (pronounced "RwaKuv") is an open-source neural network architecture that combines the parallelizable training of Transformers with the constant-time
Recurrent Neural Network
A recurrent neural network (RNN) is a neural network whose computation includes a state that is passed from one step to the next.
Self-attention
Self-attention is a neural network operation in which each position forms a data-dependent mixture of information from positions in the same input sequence or set.
Sparse attention
Sparse attention is a family of techniques that cut the computational and memory cost of the attention mechanism in transformer models by letting each token attend to only a subset of other tokens in a sequence
Tower
In deep learning, a tower is a self-contained sub-network inside a larger model that encodes one specific input or feature group into an output, usually an embedding