Masked autoencoder (MAE)
Masked autoencoder (MAE) is a self-supervised learning method for vision transformers that masks roughly 75% of an input image's patches and trains a network to reconstruct the missing pixels from the small…
Training & Optimization
Explore Training & Optimization through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Transformer Models
Articles that also belong to these categories. Counts cover all of Training & Optimization.
All articles (145)Machine Learning (70)Deep Learning (41)Large Language Models (18)AI Infrastructure (9)Mathematics (9)AI Inference (8)Reinforcement Learning (8)AI Alignment (7)Developer Tools (7)Natural Language Processing (7)Open Source AI (7)MLOps (6)Statistics (6)NVIDIA (4)AI Hardware (3)Artificial Intelligence (3)Computer Vision (3)Reasoning Models (3)Algorithms (2)Chinese AI (2)Data & Datasets (2)Generative AI (2)Meta AI (2)Model Evaluation (2)Robotics (2)Transformer Models (2)AI Agents (1)AI Code Generation (1)AI Models (1)AI Research (1)AI Tools & Products (1)Enterprise AI (1)Google (1)Google DeepMind (1)Humanoid Robots (1)Microsoft (1)Mixture of Experts (1)Model Architecture (1)Neural Networks (1)OpenAI (1)Small Language Models (1)World Models (1)
Showing 1-2 of 2 articles
Ring Attention
Ring Attention, formally Ring Attention with Blockwise Transformers, is a distributed algorithm for computing the self-attention operation of transformer neural networks across a ring of compute devices