Disaggregated serving
Disaggregated serving is an LLM inference architecture that physically separates the prefill phase and the decode phase of text generation onto different sets of GPU hardware.
AI Inference
Explore AI Inference through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Artificial Intelligence
Articles that also belong to these categories. Counts cover all of AI Inference.
All articles (97)AI Hardware (30)AI Infrastructure (26)Large Language Models (16)Developer Tools (15)NVIDIA (11)Machine Learning (10)Deep Learning (9)Open Source AI (9)AI Companies (8)Training & Optimization (8)MLOps (6)Model Architecture (4)Reasoning Models (4)Algorithms (3)Chinese AI (3)Reinforcement Learning (3)AI Alignment (2)AI Models (2)Anthropic (2)Artificial Intelligence (2)Data Centers (2)Edge computing (2)Google (2)Information Retrieval (2)Meta AI (2)AI Benchmarks (1)AI Research (1)AI Safety (1)Computer Science (1)Enterprise AI (1)Generative AI (1)Image Generation (1)Microsoft (1)Mixture of Experts (1)Natural Language Processing (1)Prompt Engineering (1)Small Language Models (1)Speech & Audio AI (1)Statistics (1)Transformer Models (1)Video Generation (1)
Showing 1-2 of 2 articles
Inference-time scaling
Inference-time scaling (also called test-time compute scaling) is the practice of improving an AI model's output quality by allocating more computational resources during inference rather than during training.