BitNet
BitNet is a family of large language model architectures developed by Microsoft Research Asia that constrain the weights of a transformer to extremely low bit-widths: initially a single bit ({-1, +1}) and…
Large Language Models
Explore language models, how they work, and the techniques used to build applications with them.
Explore articles
Browse subtopics: Model Architecture
Articles that also belong to these categories. Counts cover all of Large Language Models.
All articles (608)AI Models (211)Open Source AI (184)Chinese AI (109)Natural Language Processing (60)Mixture of Experts (55)Reasoning Models (53)Machine Learning (52)Multimodal AI (51)AI Companies (40)OpenAI (39)Artificial Intelligence (37)AI Benchmarks (35)Anthropic (34)AI Agents (30)AI Safety (29)Google DeepMind (28)Developer Tools (25)AI Inference (24)AI Code Generation (22)Meta AI (22)Small Language Models (20)Training & Optimization (18)Generative AI (16)AI Research (15)Model Architecture (15)NVIDIA (15)Model Evaluation (13)Prompt Engineering (13)Data & Datasets (12)Deep Learning (12)Google (12)AI Tools & Products (9)Conversational AI (9)Microsoft (9)Diffusion Models (8)Transformer Models (8)AI Infrastructure (7)Enterprise AI (7)Interpretability (6)People (6)Research Organizations (6)AI Alignment (5)AI for Science (5)Information Retrieval (5)AI Hardware (4)Healthcare AI (4)Reinforcement Learning (4)AI Policy & Regulation (3)AI in Cybersecurity (3)Legal AI (3)Mathematics (3)Speech & Audio AI (3)AI History (2)Algorithms (2)Computer Vision (2)Neural Networks (2)AI in Gaming (1)Autonomous Vehicles (1)ChatGPT (1)Drug Discovery (1)MLOps (1)Robotics (1)Software Development (1)
Showing 1-15 of 15 articles
BitNet b1.58
BitNet b1.58 is a ternary-weight large language model architecture from Microsoft Research in which every weight is constrained to one of three values, -1, 0, or +1
Byte Latent Transformer
The Byte Latent Transformer (BLT) is a tokenizer-free large language model architecture introduced by researchers at Meta AI's Fundamental AI Research (FAIR) group in December 2024.
ERNIE 5.5 Flash Base
ERNIE 5.5 Flash Base is a text-only large language model released by Baidu on November 5, 2026.
GLM
GLM, short for General Language Model, is both a pretraining framework for language understanding and generation and the name of a model family developed by researchers associated with Tsinghua University and…
Jamba
Jamba is a family of open-weight large language models from AI21 Labs, first released on March 28, 2024, and is the world's first production-grade language model built on a Mamba state space model (SSM)…
Jamba2
Jamba2 is the second generation of hybrid State Space Model and Transformer language models released by AI21 Labs on January 8, 2026.
Kimi Linear
Kimi Linear is a hybrid linear attention architecture published by Moonshot AI on October 30, 2025, together with a 48-billion-parameter mixture-of-experts model that activates 3 billion parameters per token.
Long-context language models
Long-context language models are large language models engineered to accept and reason over inputs far larger than the few-thousand-token windows used by early transformer systems, with frontier models in 2026…
LongRoPE
LongRoPE is a context-window extension technique for large language models (LLMs) that use rotary position embeddings (RoPE).
Mamba
Mamba is a neural network architecture for sequence modeling that uses selective state space models (SSMs) to process sequential data in linear time with respect to sequence length
Native Sparse Attention (NSA)
Native Sparse Attention (NSA) is a hardware-aligned, natively trainable sparse attention mechanism introduced in February 2025 by researchers at DeepSeek, Peking University and the University of Washington.
SparDA
SparDA (Sparse Decoupled Attention) is an add-on architecture for long-context large language model inference proposed by researchers at NVIDIA in a paper posted to arXiv on 3 June 2026.
SubQ
SubQ is a family of large language models and decision models built by Subquadratic, an AI startup that emerged from stealth on May 5, 2026
YOCO (You Only Cache Once)
YOCO ("You Only Cache Once") is a decoder-decoder neural network architecture for large language models introduced by researchers at Microsoft Research and Tsinghua University in May 2024.