BitNet
BitNet is a family of large language model architectures developed by Microsoft Research Asia that constrain the weights of a transformer to extremely low bit-widths: initially a single bit ({-1, +1}) and…
Large Language Models
Explore language models, how they work, and the techniques used to build applications with them.
Explore articles
Browse subtopics: Model Architecture
Articles that also belong to these categories. Counts cover all of Large Language Models.
All articles (545)AI Models (184)Open Source AI (165)Chinese AI (92)Natural Language Processing (58)AI Companies (53)Machine Learning (49)Mixture of Experts (46)Multimodal AI (41)Reasoning Models (39)Artificial Intelligence (37)OpenAI (35)AI Benchmarks (33)Anthropic (31)Google DeepMind (26)AI Agents (22)AI Safety (21)Developer Tools (20)Meta AI (20)AI Code Generation (18)Small Language Models (18)Training & Optimization (18)AI Inference (16)Model Architecture (14)Generative AI (13)NVIDIA (13)Prompt Engineering (13)AI Research (12)Deep Learning (12)Google (12)Model Evaluation (12)Data & Datasets (11)AI Tools & Products (9)Conversational AI (9)Microsoft (9)Transformer Models (9)AI Infrastructure (7)Diffusion Models (7)AI Alignment (5)Enterprise AI (5)Research Organizations (5)AI Hardware (4)Healthcare AI (4)Information Retrieval (4)Interpretability (4)People (4)Mathematics (3)AI History (2)AI for Science (2)Algorithms (2)Computer Vision (2)Legal AI (2)Reinforcement Learning (2)Speech & Audio AI (2)AI Policy (1)AI Policy & Regulation (1)AI in Cybersecurity (1)AI in Gaming (1)Autonomous Vehicles (1)ChatGPT (1)China AI (1)Drug Discovery (1)Neural Networks (1)Robotics (1)Software Development (1)Video Generation (1)
Showing 1-14 of 14 articles
BitNet b1.58
BitNet b1.58 is a ternary-weight large language model architecture from Microsoft Research in which every weight is constrained to one of three values, -1, 0, or +1
Byte Latent Transformer
The Byte Latent Transformer (BLT) is a tokenizer-free large language model architecture introduced by researchers at Meta AI's Fundamental AI Research (FAIR) group in December 2024.
GLM
GLM, short for General Language Model, is both a pretraining framework for language understanding and generation and the name of a model family developed by researchers associated with Tsinghua University and…
Jamba
Jamba is a family of open-weight large language models from AI21 Labs, first released on March 28, 2024, and is the world's first production-grade language model built on a Mamba state space model (SSM)…
Jamba2
Jamba2 is the second generation of hybrid State Space Model and Transformer language models released by AI21 Labs on January 8, 2026.
Kimi Linear
Kimi Linear is a hybrid linear attention architecture published by Moonshot AI on October 30, 2025, together with a 48-billion-parameter mixture-of-experts model that activates 3 billion parameters per token.
Long-context language models
Long-context language models are large language models engineered to accept and reason over inputs far larger than the few-thousand-token windows used by early transformer systems, with frontier models in 2026…
LongRoPE
LongRoPE is a context-window extension technique for large language models (LLMs) that use rotary position embeddings (RoPE).
Mamba
Mamba is a neural network architecture for sequence modeling that uses selective state space models (SSMs) to process sequential data in linear time with respect to sequence length
Rotary Position Embedding
Rotary Position Embedding (RoPE) is a positional encoding method for transformer models that encodes a token's absolute position by rotating its query and key vectors in two-dimensional subspaces
SparDA
SparDA (Sparse Decoupled Attention) is an add-on architecture for long-context large language model inference proposed by researchers at NVIDIA in a paper posted to arXiv on 3 June 2026.
SubQ
SubQ is a large language model released on May 5, 2026 by Subquadratic, a Miami-based startup that emerged from stealth claiming to have built the first frontier model on a fully subquadratic attention…
YOCO (You Only Cache Once)
YOCO ("You Only Cache Once") is a decoder-decoder neural network architecture for large language models introduced by researchers at Microsoft Research and Tsinghua University in May 2024.