Showing 1-36 of 36 articles
DBRX
DBRX is an open-weight mixture of experts large language model developed by Databricks and its Mosaic AI research team, released on March 27, 2024. It has 132...
AI CompaniesLarge Language Models
DeepEP
DeepEP is an open-source GPU communication library built by DeepSeek for Mixture-of-Experts (MoE) models. It provides fast all-to-all kernels for the two...
AI InfrastructureOpen Source AI
DeepSeek V3
DeepSeek-V3 is a 671-billion-parameter open-weights Mixture of Experts large language model from Chinese AI lab DeepSeek, released on December 26, 2024, that...
AI ModelsChinese AI
DeepSeek V3.1
DeepSeek V3.1 is a large language model developed by DeepSeek, released on August 19, 2025 and made broadly available via the official API on August 21,...
AI ModelsChinese AI
DeepSeek V4
DeepSeek V4 is a family of open-weight Mixture of Experts large language models developed by DeepSeek, a Hangzhou-based AI research lab. Released in preview on...
AI ModelsChinese AI
DeepSeek-V2
DeepSeek-V2 is a 236-billion-parameter mixture-of-experts (MoE) large language model released in May 2024 by DeepSeek, the Chinese AI lab spun out of the...
Chinese AILarge Language Models
DeepSeek-VL2
DeepSeek-VL2 is an open-weights family of Mixture-of-Experts (MoE) vision-language models released by the Chinese AI laboratory DeepSeek on December 13,...
Chinese AIMultimodal AI
Dynamic inference
Dynamic inference, also called input-adaptive inference, conditional computation, or adaptive computation, is the family of techniques that adapts a neural...
AI Inference
Expert Parallelism
Expert Parallelism (EP) is a model-parallelism strategy specific to Mixture of Experts (MoE) neural networks in which the individual expert sub-networks...
AI InfrastructureTraining & Optimization
GLM-4.5
GLM-4.5 is an open-weights large language model released by Zhipu AI (operating internationally as Z.ai) on July 28, 2025, built on a 355-billion-parameter...
AI ModelsChinese AI
GLM-4.6
GLM-4.6 is a flagship open-weight large language model released by Zhipu AI under its international brand Z.ai on September 30, 2025, built on a sparse Mixture...
AI ModelsChinese AI
GLM-5
GLM-5 is an open-weight flagship large language model released by the Chinese AI company Zhipu AI, under its international brand Z.ai, on February 11, 2026. It...
AI ModelsChinese AI
Hunyuan
Hunyuan is Tencent's brand for its foundation models, an umbrella covering large language models, video generation, image synthesis, 3D asset creation, and...
AI ModelsChinese AI
Hunyuan-A13B
Hunyuan-A13B is an open-weight mixture-of-experts large language model released by Tencent in late June 2025. It has about 80 billion total parameters but...
Chinese AILarge Language Models
Jamba
Jamba is a family of open-weight large language models from AI21 Labs, first released on March 28, 2024, and is the world's first production-grade language...
AI CompaniesLarge Language Models
Jamba2
Jamba2 is the second generation of hybrid State Space Model and Transformer language models released by AI21 Labs on January 8, 2026.[1] The family extends the...
AI ModelsLarge Language Models
Kimi K2
Kimi K2 is an open-weights Mixture of Experts language model from Moonshot AI, a Beijing startup, released on July 11, 2025 with 1.04 trillion total parameters...
AI ModelsChinese AI
Kimi K2.5
Kimi K2.5 is an open-weights, natively multimodal large language model developed by Moonshot AI and released on January 27, 2026 [4][9]. It extends the...
AI ModelsChinese AI
Ling-1T
Ling-1T is a trillion-parameter open-weight language model released by Ant Group through its Inclusion AI research group on October 9, 2025. It is a mixture of...
Chinese AILarge Language Models
Llama 4 Behemoth
Llama 4 Behemoth is the announced but unreleased flagship model in the Llama 4 family from Meta AI. It was unveiled on April 5, 2025 alongside its smaller...
AI ModelsLarge Language Models
Llama 4 Scout and Maverick
Llama 4 Scout and Llama 4 Maverick are open-weight, natively multimodal AI large language models developed by Meta and released on April 5, 2025.[1] They are...
AI ModelsLarge Language Models
MiniMax M2
MiniMax M2 is an open-weight large language model released on October 27, 2025 by the Shanghai-based AI company MiniMax, built as a Mixture of Experts model...
AI AgentsAI Models
MiniMax-Text-01
MiniMax-Text-01 is an open-weights, large-scale mixture-of-experts (MoE) language model released by Shanghai-based AI company MiniMax on January 14,...
Chinese AILarge Language Models
Mistral Large 3
Mistral Large 3 is a sparse mixture-of-experts large language model released on December 2, 2025 by the French AI company Mistral AI, distributed as open...
AI CompaniesAI Models
Mixtral
Mixtral is a family of open-weight Sparse Mixture of Experts (SMoE) large language models developed by Mistral AI, a French artificial intelligence company...
AI CompaniesLarge Language Models
Mixtral 8x22B
Mixtral 8x22B is a sparse mixture-of-experts (MoE) large language model released by the French AI company Mistral AI on April 17, 2024. It has approximately...
AI CompaniesAI Models
OLMoE
OLMoE (Open Mixture-of-Experts) is a fully open sparse mixture of experts large language model released by the Allen Institute for AI (Ai2) on September 3,...
AI ModelsLarge Language Models
Qwen3
Qwen3 is the third-generation family of large language models developed by the Qwen Team at Alibaba Cloud (also known as Tongyi Qianwen lab). Released on April...
AI ModelsChinese AI
Qwen3-Max
Qwen3-Max is the flagship large language model in Alibaba's Qwen series and the first Qwen model to cross one trillion parameters, released in preview on...
AI ModelsChinese AI
Snowflake Arctic
Snowflake Arctic is an open large language model released by Snowflake AI Research on April 24, 2024.[1][2][10] It uses a Dense Mixture of Experts (Dense-MoE)...
AI CompaniesLarge Language Models
Switch Transformer
The Switch Transformer is a sparsely activated Mixture of Experts (MoE) Transformer architecture introduced by William Fedus, Barret Zoph, and Noam Shazeer at...
GoogleLarge Language Models
Tencent Hunyuan Hy3
Tencent Hunyuan Hy3 (released as Hy3-preview) is an open-weights large language model published by Tencent on April 23, 2026. It is a Mixture-of-Experts (MoE)...
Chinese AILarge Language Models
Yi-Lightning
Yi-Lightning is a closed-source large language model developed by Chinese artificial intelligence company 01.AI (零一万物, Língyī Wànwù), the company founded by...
AI ModelsChinese AI
ZAYA1-8B
ZAYA1-8B is an open-weight, reasoning-focused Mixture-of-Experts (MoE) large language model released by San Francisco-based AI research lab Zyphra on May 6,...
AI ModelsLarge Language Models
Zyphra
Zyphra is an American artificial intelligence research and product company headquartered in San Francisco, California, with a secondary office in London.[1]...
AI CompaniesLarge Language Models
gpt-oss
gpt-oss is a family of open-weight large language models released by OpenAI on August 5, 2025, and OpenAI's first open-weight language models since GPT-2 in...
AI ModelsLarge Language Models