Flash-Decoding
Flash-Decoding is an inference-time variant of the FlashAttention algorithm that targets the decoding (autoregressive generation) phase of large language model inference
Algorithms
Explore Algorithms through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: AI Inference
Articles that also belong to these categories. Counts cover all of Algorithms.
All articles (37)Machine Learning (18)Reinforcement Learning (8)Statistics (7)Deep Learning (4)Robotics (4)AI Hardware (3)AI Inference (3)Computer Science (3)Information Retrieval (3)Mathematics (3)Natural Language Processing (3)Open Source AI (3)Artificial Intelligence (2)Large Language Models (2)Training & Optimization (2)AI for Science (1)AI in Gaming (1)Computer Vision (1)Data & Datasets (1)Google (1)Google DeepMind (1)Interpretability (1)Transformer Models (1)
Showing 1-3 of 3 articles
Lookahead Decoding
Lookahead Decoding is a parallel decoding algorithm for accelerating inference in large language models, introduced in November 2023 by Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang from the Hao AI Lab at…
Top-k sampling
Top-k sampling is a decoding strategy for autoregressive language models that restricts each generation step to the k most probable next tokens.