Switch Transformer
The Switch Transformer is a sparsely activated Mixture of Experts (MoE) Transformer architecture introduced by William Fedus, Barret Zoph, and Noam Shazeer at Google in January 2021.
Explore Google through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Mixture of Experts
Articles that also belong to these categories. Counts cover all of Google.
All articles (102)AI Hardware (24)Machine Learning (16)AI Tools & Products (14)AI Infrastructure (13)Large Language Models (12)AI Models (11)Developer Tools (11)Open Source AI (11)Artificial Intelligence (7)Generative AI (7)People (7)Google DeepMind (6)Multimodal AI (6)AI Agents (5)AI Code Generation (5)AI History (5)Conversational AI (5)Image Generation (5)AI Research (4)Healthcare AI (4)Speech & Audio AI (4)AI Companies (3)AI for Science (3)Computer Vision (3)Deep Learning (3)Enterprise AI (3)Music & Audio Generation (3)AI Inference (2)Data Centers (2)Model Architecture (2)Small Language Models (2)Software Development (2)Transformer Models (2)AI Safety (1)Algorithms (1)Anthropic (1)Autonomous Vehicles (1)Computer Science (1)Data & Datasets (1)Diffusion Models (1)Drug Discovery (1)Edge computing (1)Education AI (1)MLOps (1)Mathematics (1)Mixture of Experts (1)Neural Networks (1)Reinforcement Learning (1)Robotics (1)Training & Optimization (1)Video Generation (1)Voice AI (1)
Showing 1-1 of 1 article