Switch Transformer
The Switch Transformer is a sparsely activated Mixture of Experts (MoE) Transformer architecture introduced by William Fedus, Barret Zoph, and Noam Shazeer at Google in January 2021.
Mixture of Experts
Explore Mixture of Experts through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Google
Articles that also belong to these categories. Counts cover all of Mixture of Experts.
All articles (53)Large Language Models (46)Open Source AI (40)AI Models (35)Chinese AI (28)Multimodal AI (8)AI Companies (7)AI Infrastructure (3)Transformer Models (3)Diffusion Models (2)Meta AI (2)Model Architecture (2)NVIDIA (2)Reasoning Models (2)AI Agents (1)AI Code Generation (1)AI Inference (1)Generative AI (1)Google (1)Google DeepMind (1)Healthcare AI (1)Natural Language Processing (1)OpenAI (1)Research Organizations (1)Training & Optimization (1)Video Generation (1)
Showing 1-1 of 1 article