Infini-Attention
Infini-attention is an attention mechanism introduced by Google researchers Tsendsuren Munkhdalai, Manaal Faruqui, and Siddharth Gopal in the April 2024 paper "Leave No Context Behind: Efficient Infinite…
Explore Google through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Model Architecture
Articles that also belong to these categories. Counts cover all of Google.
All articles (102)AI Hardware (24)Machine Learning (16)AI Tools & Products (14)AI Infrastructure (13)Large Language Models (12)AI Models (11)Developer Tools (11)Open Source AI (11)Artificial Intelligence (7)Generative AI (7)People (7)Google DeepMind (6)Multimodal AI (6)AI Agents (5)AI Code Generation (5)AI History (5)Conversational AI (5)Image Generation (5)AI Research (4)Healthcare AI (4)Speech & Audio AI (4)AI Companies (3)AI for Science (3)Computer Vision (3)Deep Learning (3)Enterprise AI (3)Music & Audio Generation (3)AI Inference (2)Data Centers (2)Model Architecture (2)Small Language Models (2)Software Development (2)Transformer Models (2)AI Safety (1)Algorithms (1)Anthropic (1)Autonomous Vehicles (1)Computer Science (1)Data & Datasets (1)Diffusion Models (1)Drug Discovery (1)Edge computing (1)Education AI (1)MLOps (1)Mathematics (1)Mixture of Experts (1)Neural Networks (1)Reinforcement Learning (1)Robotics (1)Training & Optimization (1)Video Generation (1)Voice AI (1)
Showing 1-2 of 2 articles
Titans (neural architecture)
Titans is a family of neural sequence-modeling architectures from Google Research that combines an attention-based "short-term memory" with a deep neural "long-term memory" module that updates its own…