Compass (Shopee language model)
Compass is a proprietary family of large language models developed by Shopee and its parent company, Sea, for Southeast Asian languages and e-commerce tasks.
Mixture of Experts
Explore Mixture of Experts through related topics and the articles other pages reference most.
Most referenced in this topic
Ranked by links from other AI Wiki pages.
Explore articles
Browse subtopics (25)
Articles that also belong to these categories. Counts cover all of Mixture of Experts.
All articles (53)Large Language Models (46)Open Source AI (40)AI Models (35)Chinese AI (28)Multimodal AI (8)AI Companies (7)AI Infrastructure (3)Transformer Models (3)Diffusion Models (2)Meta AI (2)Model Architecture (2)NVIDIA (2)Reasoning Models (2)AI Agents (1)AI Code Generation (1)AI Inference (1)Generative AI (1)Google (1)Google DeepMind (1)Healthcare AI (1)Natural Language Processing (1)OpenAI (1)Research Organizations (1)Training & Optimization (1)Video Generation (1)
Showing 1-53 of 53 articles
DBRX
DBRX is an open-weight mixture of experts large language model developed by Databricks and its Mosaic AI research team, released on March 27, 2024.
DeepEP
DeepEP is an open-source GPU communication library built by DeepSeek for Mixture-of-Experts (MoE) models.
DeepSeek V3
DeepSeek-V3 is a 671-billion-parameter open-weights Mixture of Experts large language model from Chinese AI lab DeepSeek, released on December 26, 2024, that activates only 37 billion parameters per token and…
DeepSeek V3.1
DeepSeek V3.1 is a large language model developed by DeepSeek, released on August 19, 2025 and made broadly available via the official API on August 21, 2025.
DeepSeek V4
DeepSeek V4 is a family of open-weight Mixture of Experts large language models developed by DeepSeek, a Hangzhou-based AI research lab.
DeepSeek V4-Pro
DeepSeek V4-Pro is the flagship model of the DeepSeek V4 family: a Mixture of Experts large language model with 1.6 trillion total parameters, 49 billion of them activated per token, and a one-million-token…
DeepSeek V4.1-Flash
DeepSeek V4.1-Flash is an open-weight multimodal mixture-of-experts model released by DeepSeek on September 10, 2026. It accepts text and images and generates text.
DeepSeek-V2
DeepSeek-V2 is a 236-billion-parameter mixture-of-experts (MoE) large language model released in May 2024 by DeepSeek, the Chinese AI lab spun out of the quantitative hedge fund High-Flyer and led by Liang…
DeepSeek-VL2
DeepSeek-VL2 is an open-weights family of Mixture-of-Experts (MoE) vision-language models released by the Chinese AI laboratory DeepSeek on December 13, 2024.
DiffusionGemma
DiffusionGemma is an experimental open-weight large language model developed by Google DeepMind for multimodal text generation through discrete diffusion.
Dynamic inference
Dynamic inference, also called input-adaptive inference, conditional computation, or adaptive computation
Expert Parallelism
Expert Parallelism (EP) is a model-parallelism strategy specific to Mixture of Experts (MoE) neural networks in which the individual expert sub-networks (typically feed-forward blocks) are sharded across…
GLM-4.5
GLM-4.5 is an open-weights large language model released by Zhipu AI (operating internationally as Z.ai) on July 28, 2025, built on a 355-billion-parameter Mixture of Experts architecture that activates 32…
GLM-4.6
GLM-4.6 is a flagship open-weight large language model released by Zhipu AI under its international brand Z.ai on September 30, 2025, built on a sparse Mixture of Experts (MoE) architecture with roughly 357…
GLM-5
GLM-5 is an open-weight flagship large language model released by the Chinese AI company Zhipu AI, under its international brand Z.ai, on February 11, 2026.
GLM-5.3-Flash
GLM-5.3-Flash is an open-weight, natively multimodal mixture-of-experts large language model released by Z.ai on August 26, 2026.
Hunyuan
Hunyuan is Tencent's brand for its foundation models, an umbrella covering large language models, video generation, image synthesis, 3D asset creation, and multimodal reasoning.
Hunyuan-A13B
Hunyuan-A13B is an open-weight mixture-of-experts large language model released by Tencent in late June 2025.
Hy4 Preview
Hy4 Preview is an open-weight large language model released by the Tencent Hy Team on August 28, 2026.
Jamba
Jamba is a family of open-weight large language models from AI21 Labs, first released on March 28, 2024, and is the world's first production-grade language model built on a Mamba state space model (SSM)…
Jamba2
Jamba2 is the second generation of hybrid State Space Model and Transformer language models released by AI21 Labs on January 8, 2026.
Kimi K2
Kimi K2 is an open-weights Mixture of Experts language model from Moonshot AI, a Beijing startup, released on July 11, 2025 with 1.04 trillion total parameters and 32.6 billion activated per token
Kimi K2.5
Kimi K2.5 is an open-weights, natively multimodal large language model developed by Moonshot AI and released on January 27, 2026 .
Kimi K3
Kimi K3 is an open-weight multimodal reasoning model developed by Moonshot AI. Moonshot made the model available through its hosted products on July 16, 2026 and published the weights, code, configuration…
Leanstral 1.5
Leanstral 1.5 is an open-weight model from Mistral AI for formal proof engineering in Lean 4.
Ling-1T
Ling-1T is a trillion-parameter open-weight language model released by Ant Group through its Inclusion AI research group on October 9, 2025.
Ling-3.0-flash
Ling-3.0-flash is an open-weight mixture-of-experts language model from inclusionAI, the open-source AI initiative of Ant Group, and the first model of the Ling 3.0 generation.
Ling-3.0-flash-Sante
Ling-3.0-flash-Sante is a health and medicine variant of Ling-3.0-flash, the 124-billion-parameter mixture-of-experts language model from inclusionAI, the open-source AI initiative of Ant Group.
Llama 4 Behemoth
Llama 4 Behemoth is the announced but never publicly released flagship model in the Llama 4 family from Meta AI.
Llama 4 Scout and Maverick
Llama 4 Scout and Llama 4 Maverick are open-weight, natively multimodal AI large language models developed by Meta and released on April 5, 2025.
MAGI-2 Preview
MAGI-2 Preview is a public research release of a unified audio-video generation model developed by Sand.ai.
MiniMax M2
MiniMax M2 is an open-weight large language model released on October 27, 2025 by the Shanghai-based AI company MiniMax, built as a Mixture of Experts model with 230 billion total parameters and roughly 10…
MiniMax-Text-01
MiniMax-Text-01 is an open-weights, large-scale mixture-of-experts (MoE) language model released by Shanghai-based AI company MiniMax on January 14, 2025.
Mistral Large 3
Mistral Large 3 is a sparse mixture-of-experts large language model released on December 2, 2025 by the French AI company Mistral AI, distributed as open weights under the Apache 2.0 license with roughly 675…
Mixtral
Mixtral is a family of open-weight Sparse Mixture of Experts (SMoE) large language models developed by Mistral AI, a French artificial intelligence company founded in April 2023.
Mixtral 8x22B
Mixtral 8x22B is a sparse mixture-of-experts (MoE) large language model released by the French AI company Mistral AI on April 17, 2024.
MoonEP
MoonEP is an open-source expert-parallel communication library for mixture-of-experts models, released by Moonshot AI on July 27, 2026 under the MIT License.
Nemotron 3.5 Lightning
NVIDIA Nemotron 3.5 Lightning is an open-weights 30 billion parameter mixture-of-experts language model with 3 billion active parameters per token, released by NVIDIA on August 11
Nemotron-Labs-TwoTower
Nemotron-Labs-TwoTower is an open-weight diffusion language model released by NVIDIA in mid-2026.
North Mini Code
North Mini Code is an open-weight large language model developed by Cohere for agentic software development and AI code generation.
OLMoE
OLMoE (Open Mixture-of-Experts) is a fully open sparse mixture of experts large language model released by the Allen Institute for AI (Ai2) on September 3, 2024 .
Qwen3
Qwen3 is the third-generation family of large language models developed by the Qwen Team at Alibaba Cloud (also known as Tongyi Qianwen lab).
Qwen3-Max
Qwen3-Max is the flagship large language model in Alibaba's Qwen series and the first Qwen model to cross one trillion parameters, released in preview on September 5, 2025 and formally launched at the Apsara…
Qwen3.8-Flash-Next
Qwen3.8-Flash-Next is an experimental open-weight multimodal large language model released by Alibaba Group's Qwen team on August 26, 2026.
Snowflake Arctic
Snowflake Arctic is an open large language model released by Snowflake AI Research on April 24, 2024.
Switch Transformer
The Switch Transformer is a sparsely activated Mixture of Experts (MoE) Transformer architecture introduced by William Fedus, Barret Zoph, and Noam Shazeer at Google in January 2021.
Tencent Hunyuan Hy3
Tencent Hunyuan Hy3 (marketed internationally as Tencent Hy3) is an open-weights large language model published by Tencent.
Yi-Lightning
Yi-Lightning is a closed-source large language model developed by Chinese artificial intelligence company 01.AI (零一万物, Língyī Wànwù), the company founded by Kai-Fu Lee.
ZAYA1-8B
ZAYA1-8B is an open-weight, reasoning-focused Mixture-of-Experts (MoE) large language model released by San Francisco-based AI research lab Zyphra on May 6, 2026.
Zyphra
Zyphra is an American artificial intelligence research and product company headquartered in San Francisco, California, with a secondary office in London.
dots3-note Preview
dots3-note Preview is an open-weight multimodal model and large language model developed by dots studio, an AI team at Xiaohongshu.
gpt-oss
gpt-oss is a family of open-weight large language models released by OpenAI on August 5, 2025, and OpenAI's first open-weight language models since GPT-2 in 2019.