Native Sparse Attention (NSA)
Native Sparse Attention (NSA) is a hardware-aligned, natively trainable sparse attention mechanism introduced in February 2025 by researchers at DeepSeek, Peking University and the University of Washington.
Large Language Models
Explore language models, how they work, and the techniques used to build applications with them.
Explore articles
Browse subtopics: Neural Networks
Articles that also belong to these categories. Counts cover all of Large Language Models.
All articles (608)AI Models (211)Open Source AI (184)Chinese AI (109)Natural Language Processing (60)Mixture of Experts (55)Reasoning Models (53)Machine Learning (52)Multimodal AI (51)AI Companies (40)OpenAI (39)Artificial Intelligence (37)AI Benchmarks (35)Anthropic (34)AI Agents (30)AI Safety (29)Google DeepMind (28)Developer Tools (25)AI Inference (24)AI Code Generation (22)Meta AI (22)Small Language Models (20)Training & Optimization (18)Generative AI (16)AI Research (15)Model Architecture (15)NVIDIA (15)Model Evaluation (13)Prompt Engineering (13)Data & Datasets (12)Deep Learning (12)Google (12)AI Tools & Products (9)Conversational AI (9)Microsoft (9)Diffusion Models (8)Transformer Models (8)AI Infrastructure (7)Enterprise AI (7)Interpretability (6)People (6)Research Organizations (6)AI Alignment (5)AI for Science (5)Information Retrieval (5)AI Hardware (4)Healthcare AI (4)Reinforcement Learning (4)AI Policy & Regulation (3)AI in Cybersecurity (3)Legal AI (3)Mathematics (3)Speech & Audio AI (3)AI History (2)Algorithms (2)Computer Vision (2)Neural Networks (2)AI in Gaming (1)Autonomous Vehicles (1)ChatGPT (1)Drug Discovery (1)MLOps (1)Robotics (1)Software Development (1)
Showing 1-2 of 2 articles
RWKV-7 (Goose)
RWKV-7, codenamed Goose, is an attention-free, RNN-style large-language-model architecture introduced in March 2025 that runs inference in linear time with constant memory per token while still training in…