FP4 (4-bit floating point)
FP4 (4-bit floating point) is a numerical format that stores a real number in just 4 bits, the smallest floating-point type in mainstream use for deep learning.
Explore AI Inference through related topics and the articles other pages reference most.
Articles that also belong to these categories. Counts cover all of AI Inference.
Showing 1-9 of 9 articles
FP4 (4-bit floating point) is a numerical format that stores a real number in just 4 bits, the smallest floating-point type in mainstream use for deep learning.
FP8 is a family of 8-bit floating-point number formats used to accelerate deep learning training and inference, built around two encodings, E4M3 (4 exponent bits, 3 mantissa bits) and E5M2 (5 exponent bits, 2…
Group Relative Policy Optimization (GRPO) is a reinforcement learning algorithm for fine-tuning large language models that eliminates the separate critic (value) network used by PPO
KTO (Kahneman-Tversky Optimization) is a method for aligning large language models with human feedback using only a binary signal of whether a model output is desirable or undesirable, rather than the paired…
MXFP4 (Microscaling FP4) is a 4-bit block-scaled number format for quantization in machine learning, defined in the Open Compute Project (OCP) Microscaling Formats (MX) Specification v1.0 of September 2023.
NormalFloat 4-bit (NF4) is a 4-bit numerical data type for storing the weights of deep neural networks, introduced in the 2023 QLoRA paper by Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer…
Pruning is a family of techniques used in machine learning and artificial intelligence to remove parts of a model or search space that are estimated to be unnecessary for accuracy or optimality.
QLoRA (Quantized Low-Rank Adaptation) is a parameter-efficient fine-tuning method that finetunes a 65-billion-parameter large language model on a single 48 GB GPU while matching the quality of full 16-bit…
Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training paradigm for large language models in which the reward signal comes from a deterministic