Model Architecture

Explore Model Architecture through related topics and the articles other pages reference most.

65 articlesSearch and filter Follow updates by RSS

Explore articles

Browse subtopics: Large Language Models

Articles that also belong to these categories. Counts cover all of Model Architecture.

Showing 1-14 of 14 articles

BitNet

BitNet is a family of large language model architectures developed by Microsoft Research Asia that constrain the weights of a transformer to extremely low bit-widths: initially a single bit ({-1, +1}) and…

Large Language Models Microsoft

BitNet b1.58

BitNet b1.58 is a ternary-weight large language model architecture from Microsoft Research in which every weight is constrained to one of three values, -1, 0, or +1

Large Language Models Microsoft

Byte Latent Transformer

The Byte Latent Transformer (BLT) is a tokenizer-free large language model architecture introduced by researchers at Meta AI's Fundamental AI Research (FAIR) group in December 2024.

Large Language Models Meta AI

GLM

GLM, short for General Language Model, is both a pretraining framework for language understanding and generation and the name of a model family developed by researchers associated with Tsinghua University and…

AI Models Chinese AI

Jamba

Jamba is a family of open-weight large language models from AI21 Labs, first released on March 28, 2024, and is the world's first production-grade language model built on a Mamba state space model (SSM)…

AI Companies Large Language Models

Jamba2

Jamba2 is the second generation of hybrid State Space Model and Transformer language models released by AI21 Labs on January 8, 2026.

AI Models Large Language Models

Kimi Linear

Kimi Linear is a hybrid linear attention architecture published by Moonshot AI on October 30, 2025, together with a 48-billion-parameter mixture-of-experts model that activates 3 billion parameters per token.

AI Models Chinese AI

Long-context language models

Long-context language models are large language models engineered to accept and reason over inputs far larger than the few-thousand-token windows used by early transformer systems, with frontier models in 2026…

Large Language Models

LongRoPE

LongRoPE is a context-window extension technique for large language models (LLMs) that use rotary position embeddings (RoPE).

Large Language Models Microsoft

Mamba

Mamba is a neural network architecture for sequence modeling that uses selective state space models (SSMs) to process sequential data in linear time with respect to sequence length

Large Language Models

Rotary Position Embedding

Rotary Position Embedding (RoPE) is a positional encoding method for transformer models that encodes a token's absolute position by rotating its query and key vectors in two-dimensional subspaces

Deep Learning Large Language Models

SparDA

SparDA (Sparse Decoupled Attention) is an add-on architecture for long-context large language model inference proposed by researchers at NVIDIA in a paper posted to arXiv on 3 June 2026.

AI Inference Large Language Models

SubQ

SubQ is a large language model released on May 5, 2026 by Subquadratic, a Miami-based startup that emerged from stealth claiming to have built the first frontier model on a fully subquadratic attention…

AI Companies Large Language Models

YOCO (You Only Cache Once)

YOCO ("You Only Cache Once") is a decoder-decoder neural network architecture for large language models introduced by researchers at Microsoft Research and Tsinghua University in May 2024.

Large Language Models Microsoft