FP4 (4-bit floating point)
FP4 (4-bit floating point) is a numerical format that stores a real number in just 4 bits, the smallest floating-point type in mainstream use for deep learning.
AI Inference
Explore AI Inference through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Training & Optimization
Articles that also belong to these categories. Counts cover all of AI Inference.
All articles (97)AI Hardware (30)AI Infrastructure (26)Large Language Models (16)Developer Tools (15)NVIDIA (11)Machine Learning (10)Deep Learning (9)Open Source AI (9)AI Companies (8)Training & Optimization (8)MLOps (6)Model Architecture (4)Reasoning Models (4)Algorithms (3)Chinese AI (3)Reinforcement Learning (3)AI Alignment (2)AI Models (2)Anthropic (2)Artificial Intelligence (2)Data Centers (2)Edge computing (2)Google (2)Information Retrieval (2)Meta AI (2)AI Benchmarks (1)AI Research (1)AI Safety (1)Computer Science (1)Enterprise AI (1)Generative AI (1)Image Generation (1)Microsoft (1)Mixture of Experts (1)Natural Language Processing (1)Prompt Engineering (1)Small Language Models (1)Speech & Audio AI (1)Statistics (1)Transformer Models (1)Video Generation (1)
Showing 1-8 of 8 articles
GRPO
Group Relative Policy Optimization (GRPO) is a reinforcement learning algorithm for fine-tuning large language models that eliminates the separate critic (value) network used by PPO
KTO
KTO (Kahneman-Tversky Optimization) is a method for aligning large language models with human feedback using only a binary signal of whether a model output is desirable or undesirable, rather than the paired…
NormalFloat 4-bit (NF4)
NormalFloat 4-bit (NF4) is a 4-bit numerical data type for storing the weights of deep neural networks, introduced in the 2023 QLoRA paper by Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer…
Pruning
Pruning is a family of techniques used in machine learning and artificial intelligence to remove parts of a model or search space that are estimated to be unnecessary for accuracy or optimality.
QLoRA
QLoRA (Quantized Low-Rank Adaptation) is a parameter-efficient fine-tuning method that finetunes a 65-billion-parameter large language model on a single 48 GB GPU while matching the quality of full 16-bit…
RLVR
Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training paradigm for large language models in which the reward signal comes from a deterministic
Test-Time Training (TTT)
Test-Time Training (TTT) is a family of machine learning techniques in which a model updates a subset of its own parameters at inference time