# MXFP4
> Source: https://aiwiki.ai/wiki/mxfp4
> Updated: 2026-10-29
> Fact-checked: 2026-10-29
> Categories: AI Hardware, AI Inference, Machine Learning, Training & Optimization
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "MXFP4." aiwiki.ai, 29 Oct 2026. https://aiwiki.ai/wiki/mxfp4
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.
**MXFP4** (Microscaling FP4) is a 4-bit block-scaled number format for [quantization](https://aiwiki.ai/wiki/quantization) in [machine learning](https://aiwiki.ai/wiki/machine_learning), defined in the [Open Compute Project](https://aiwiki.ai/wiki/open_compute_project) (OCP) Microscaling Formats (MX) Specification v1.0 of September 2023. Each MXFP4 block holds 32 values stored as 4-bit [FP4](https://aiwiki.ai/wiki/fp4) E2M1 floating-point numbers, plus one shared 8-bit E8M0 scale that can only represent a power of two. The block takes 136 bits, or 4.25 bits per value.[1][9] MXFP4 is the 4-bit member of the [microscaling formats](https://aiwiki.ai/wiki/microscaling_formats) family. It has native matrix-unit support on [NVIDIA Blackwell](https://aiwiki.ai/wiki/nvidia_blackwell) GPUs, the [AMD Instinct MI355X](https://aiwiki.ai/wiki/amd_instinct_mi355x) and later AMD accelerators, and [AWS Trainium 3](https://aiwiki.ai/wiki/aws_trainium_3).[6][17][19] OpenAI's [gpt-oss](https://aiwiki.ai/wiki/gpt_oss) models, released in August 2025, store their [mixture-of-experts](https://aiwiki.ai/wiki/mixture_of_experts) weights in MXFP4, and those weights make up more than 90% of each model's parameters.[9]
MXFP4 is not the only block-scaled FP4 format. NVIDIA's [NVFP4](https://aiwiki.ai/wiki/nvfp4) uses the same E2M1 elements but has smaller 16-value blocks, an FP8 block scale and a per-tensor FP32 scale. Published comparisons generally find NVFP4 more accurate at a slightly higher storage cost. MXFP4's advantages are that it is a cross-vendor open standard and has a simpler scale.[5][6][7]
## Format specification
The OCP specification describes an MX format with three parameters: the scale data type X, the element data type of the private elements Pi, and the scaling block size k. A block of k elements takes w + kd bits, where w is the scale width and d is the element width. Each value in the block decodes as X times Pi. The specification does not prescribe a memory layout, so a scale can be stored next to its elements or separately from them.[1]
| Parameter | MXFP4 value (OCP MX v1.0) |
|---|---|
| Element type | FP4 E2M1: 1 sign bit, 2 exponent bits, 1 mantissa bit, exponent bias 1 |
| Element range | Maximum normal ±6.0; minimum normal ±1.0; subnormal ±0.5 |
| Infinities and NaN in elements | None reserved |
| Block size (k) | 32 |
| Scale type | E8M0: unsigned 8-bit biased exponent, bias 127, exponents -127 to 127 |
| Scale special values | One NaN encoding (11111111); no zero, no infinity |
| Bits per block | 8 + 32 x 4 = 136 |
| Average storage | 4.25 bits per value |
Sources: OCP MX v1.0 specification, tables 1, 5 and 7.[1] The 4.25-bit figure is the one OpenAI gives for gpt-oss.[9]
Applying the specification's decoding formula to E2M1 (bias 1, with subnormals) gives eight non-negative magnitudes. With the sign bit, the element can represent ±0, ±0.5, ±1, ±1.5, ±2, ±3, ±4 and ±6.[1][5]
| Bits (S EE M) | Value | Kind |
|---|---|---|
| 0 00 0 | 0 | Zero |
| 0 00 1 | 0.5 | Subnormal |
| 0 01 0 | 1.0 | Normal |
| 0 01 1 | 1.5 | Normal |
| 0 10 0 | 2.0 | Normal |
| 0 10 1 | 3.0 | Normal |
| 0 11 0 | 4.0 | Normal |
| 0 11 1 | 6.0 | Normal (maximum) |
The negative values have the same pattern with the sign bit set. If the shared scale is the NaN encoding, the specification makes every value in the block NaN regardless of the element bits.[1]
### Conversion rules
The specification's reference conversion from a 32-value vector V works like this. X is set to the largest power of two that does not exceed max(|Vi|), divided by the largest power of two the element type can represent; for FP4 that divisor is 4. Each Vi/X is then quantized to FP4, and any value above the maximum normal is clamped to ±6. Implementations must support round-to-nearest-ties-to-even and saturating overflow. They must flush values below the smallest subnormal to zero, and they may offer other rounding modes.[1] The specification states that "detailed algorithms for computing the block scale" are out of scope, so other conversion schemes are allowed.[1]
For dot products, the specification requires at minimum C = X(A) X(B) Σ (Pi(A) × Pi(B)). The two shared scales are factored out, so the inner reduction operates only on the narrow elements. Internal precision and order of operations are left to the implementation. A longer dot product is the sum of these block dot products, and the specification says the result should be Float32.[1]
### Clipping in the reference algorithm
The reference scale rule can push the largest value in a block past the FP4 range. Tseng, Yu and Park point out that for MXFP4 the shared exponent works out to floor(log2(amax)) - 2. After scaling, the block maximum can therefore land anywhere just below 8, and values that scale to between 6 and 8 are clipped to 6. For example, a block whose largest magnitude is 7.5 gets a scale of 1, and 7.5 is stored as 6. The authors estimate that about 3% of entries are clipped across a wide range of matrices, which biases the result.[4] NVIDIA's NVFP4 pretraining report takes the opposite approach. It rounds MXFP4 decode scales up to the next power of two so that nothing saturates. In the worst case this leaves the ±4 and ±6 codes unused and wastes nearly one binade (a factor-of-two band) of range, using about 2.58 of FP4's 3.58 binades.[5]
## Place in the MX family
The MX v1.0 specification defines four concrete formats, all with 32-element blocks and an E8M0 scale. MXFP4 is the only one with a 4-bit element.[1] A compliant implementation does not have to support every format.[1] The microscaling formats article covers the family as a whole.
| Format | Element types | Element bits | Block size | Scale |
|---|---|---|---|---|
| MXFP8 | FP8 E5M2 or E4M3 | 8 | 32 | E8M0 |
| MXFP6 | FP6 E3M2 or E2M3 | 6 | 32 | E8M0 |
| MXFP4 | FP4 E2M1 | 4 | 32 | E8M0 |
| MXINT8 | INT8 | 8 | 32 | E8M0 |
Source: OCP MX v1.0, table 1.[1]
The specification's version table dates v1.0 to 7 September 2023, with Bita Darvish Rouhani as author. The document lists authors from AMD, Arm, Intel, Meta, [Microsoft](https://aiwiki.ai/wiki/microsoft), NVIDIA and Qualcomm.[1] The OCP Foundation announced the release on 17 October 2023. The announcement said the seven companies had formed the Microscaling Formats (MX) Alliance earlier that year and were publishing the specification "in an open, license-free format".[2] Microsoft also released microxcaling, a [PyTorch](https://aiwiki.ai/wiki/pytorch) library that emulates MX formats in software.[23]
## Hardware support
| Hardware | MXFP4 support | Source |
|---|---|---|
| NVIDIA Blackwell (data center: B200 and GB200 at compute capability 10.0, B300 and GB300 at 10.3) | Fifth-generation [Tensor Cores](https://aiwiki.ai/wiki/tensor_core) support MXFP4 with hardware-accelerated block scaling, alongside plain FP4 and NVFP4 | [6][16] |
| NVIDIA Blackwell (compute capability 12.0: GeForce RTX 50 series, RTX PRO Blackwell; 12.1: GB10 in [DGX Spark](https://aiwiki.ai/wiki/nvidia_dgx_spark)) | PTX documents the block-scaled `mma.sync` kinds `.kind::mxf4` and `.kind::mxf4nvf4` for the sm_120a and sm_121a targets | [15][16] |
| [AMD Instinct MI355X](https://aiwiki.ai/wiki/amd_instinct_mi355x) (CDNA 4) | Peak MXFP4 matrix throughput of 10.1 PFLOPS, the same as MXFP6, against 5 PFLOPS for MXFP8 | [17] |
| [AMD Instinct MI455X](https://aiwiki.ai/wiki/amd_instinct_mi455x) | Peak OCP MXFP4 throughput of 40.3 PFLOPS | [18] |
| [AWS Trainium 3](https://aiwiki.ai/wiki/aws_trainium_3) | 2,517 MXFP8/MXFP4 TFLOPS per chip; MX matmuls run at 4x BF16/FP16 throughput | [19][20] |
All figures are vendor-published peak numbers as of 30 October 2026.
In NVIDIA's PTX instruction set, `.kind::mxf4` multiplies E2M1 operands using UE8M0 block scales. `.kind::mxf4nvf4` accepts either UE8M0 or UE4M3 scales, so the same instruction family covers both MXFP4 and NVFP4.[15] NVIDIA's comparison table lists MXFP4 as having accelerated hardware scaling on Blackwell, while plain FP4 relies on a software scaling factor.[6] The authors of the Quartet training paper (May 2025) wrote that MXFP4 was then "the only microscaling format with support for all required layouts for both forward and backward multiplications in low precision on Blackwell".[8]
The AWS Neuron documentation describes Trainium 3's MX groups as 32 elements along the matmul contraction dimension, sharing one 8-bit scale. The Tensor Engine multiplies the matrices and applies the MX scales in a single instruction, with output in FP32 or BF16.[20]
## Software support
| Software | MXFP4 support | Source |
|---|---|---|
| [vLLM](https://aiwiki.ai/wiki/vllm) | Launched gpt-oss support on 5 August 2025. MXFP4 MoE runs through a [FlashInfer](https://aiwiki.ai/wiki/flashinfer) kernel that uses Blackwell's native MXFP4 tensor cores, and through the [Triton](https://aiwiki.ai/wiki/openai_triton) `matmul_ogs` kernel on Hopper GPUs | [12] |
| [llama.cpp](https://aiwiki.ai/wiki/llama_cpp) / [GGUF](https://aiwiki.ai/wiki/gguf) | PR #15091 ("llama : add gpt-oss"), merged 5 August 2025, added an MXFP4 data type to ggml with CPU, CUDA, Vulkan and Metal backends. `GGML_TYPE_MXFP4` stores 32 values as one E8M0 byte plus 16 bytes of packed nibbles | [13][14] |
| OpenAI reference code | The gpt-oss repository's reference Triton implementation uses an MoE kernel from Triton's `triton_kernels` that supports MXFP4; there is also a Metal implementation for [Apple Silicon](https://aiwiki.ai/wiki/apple_silicon) | [10] |
| [SGLang](https://aiwiki.ai/wiki/sglang) | The Ling-3.0-flash-fp4 model card launches SGLang with `--moe-runner-backend flashinfer_mxfp4` | [22] |
| [Humming](https://aiwiki.ai/wiki/humming) | Independent kernel library (`humming-kernels` on PyPI) with an MXFP4 MoE backend that compiles kernels at runtime with NVRTC | [21] |
| PyTorch torchao | Documents MXFP4 inference alongside MXFP8 and NVFP4 | [24] |
## Use in models
### gpt-oss
[OpenAI](https://aiwiki.ai/wiki/openai) post-trained gpt-oss-120b and gpt-oss-20b with their MoE weights quantized to MXFP4. Because those weights account for more than 90% of the parameters, the larger model fits on a single 80 GB GPU and the smaller one runs on systems with as little as 16 GB of memory. The model card lists checkpoint sizes of 60.8 GiB for gpt-oss-120b and 12.8 GiB for gpt-oss-20b.[9] OpenAI states that all of its published evaluations used the same MXFP4 quantization. It names the [NVIDIA H100](https://aiwiki.ai/wiki/nvidia_h100) and [AMD Instinct MI300X](https://aiwiki.ai/wiki/amd_instinct_mi300x) as examples of a single 80 GB GPU that can run the 120b model.[11]
The checkpoint stores each MXFP4 tensor in two parts. `tensor.blocks` holds the FP4 values, two per `uint8`, and `tensor.scales` holds the block scales, with blocks formed along the tensor's last dimension. All other tensors are in [bfloat16](https://aiwiki.ai/wiki/bfloat16), and OpenAI recommends BF16 activations.[10] vLLM described the arrangement as "two fp4 values are packed into a single 8-bit unit in memory, and then unpacked on the fly within the matmul kernel".[12]
### Ling-3.0-flash on DGX Spark
The `Ling-3.0-flash-fp4` checkpoint from [inclusionAI](https://aiwiki.ai/wiki/inclusionai) stores its routed experts in MXFP4: E2M1 weights, E8M0 scales and a group size of 32, converted from a 128x128 block-FP8 source. Most other layers use blockwise FP8 E4M3 with dynamic activation scaling, and the LM head and several attention projections are excluded from quantization.[22]
On 27 October 2026, an NVIDIA technical blog post used this checkpoint, a 124-billion-parameter MoE with 5.1 billion active parameters, to study batch-size-1 decoding on [NVIDIA DGX Spark](https://aiwiki.ai/wiki/nvidia_dgx_spark). Profiling showed decoding was memory-bound, with weight reads driving execution time. The baseline used the Ling cookbook's W4A8 configuration with FlashInfer MXFP4 MoE kernels.[21] Swapping the MoE backend to Humming, which builds kernels for the model's actual matrix shapes at runtime, cut time per output token from 27.71 ms to 25.35 ms. That is a 9.3% rise in decode throughput, from 36.09 to 39.45 tokens per second, on NVIDIA's synthetic 32K-input, 2K-output workload.[21] The post adds FP8 quantization of the LM head and [multi-token prediction](https://aiwiki.ai/wiki/multi_token_prediction) [speculative decoding](https://aiwiki.ai/wiki/speculative_decoding) on top. On all 1,319 GSM8K questions, the full stack raised decode throughput by 42.5%, while GSM8K accuracy moved from 96.89% to 96.82%.[21]
## Comparison with NVFP4
| Property | MXFP4 | NVFP4 |
|---|---|---|
| Element | FP4 E2M1 | FP4 E2M1 |
| Block size | 32 | 16 |
| Block scale | E8M0 (power of two) | FP8 E4M3 (fractional) |
| Second-level scale | None | FP32, per tensor |
| Average storage | 4.25 bits per value | 4.5 bits per value |
| Origin | OCP MX v1.0 open standard (2023) | NVIDIA, introduced with Blackwell |
Sources: OCP MX v1.0;[1] NVIDIA;[6] Egiazarian et al.[7]
NVIDIA's NVFP4 introduction says the E8M0 scale "snaps the scale factor to nearest 2ⁿ", which can produce a large error on the block maximum, while an E4M3 scale can be fitted more closely to each block. NVIDIA still names a reason to choose E8M0: it is simpler, it needs no extra per-tensor software scale, and it "can be adequate for activations and weights that are less sensitive to the precision of scale factors".[6] NVIDIA's comparison table rates both plain FP4 and MXFP4 as carrying a "risk of noticeable accuracy drop compared to FP8", and NVFP4 as lower-risk.[6]
Independent and vendor studies point the same way:
- **Post-training quantization.** Egiazarian, Castro, Alistarh and colleagues ran what they describe as the first comprehensive study of both formats for post-training quantization of Llama 3 and Qwen3 models. MXFP4 caused "major accuracy drops (∼ 10% relative)", which they attribute to the error introduced by power-of-two scales. Their MR-GPTQ method, a [GPTQ](https://aiwiki.ai/wiki/gptq) variant with block-wise Hadamard rotations, brought MXFP4 to within 1-2% of NVFP4, and for large models both formats recovered up to 98-99% of FP16 accuracy. The authors also report that their MXFP4 kernel can beat the throughput of an ideal NVFP4 matrix multiplication.[7]
- **Pretraining.** NVIDIA trained an 8-billion-parameter hybrid Mamba-Transformer model on 1 trillion tokens with each format. Measured against BF16, MXFP4's training loss had a relative error of about 2.5%, against 1.5% for NVFP4. MXFP4 matched NVFP4's final loss only after 36% more tokens (1.36 trillion).[5] This is a vendor comparison of its own format against the open one.
## Training with MXFP4
The paper behind the MX specification, "Microscaling Data Formats for Deep Learning" (Rouhani et al., arXiv:2310.10537), tested MXFP4 both for inference and as part of a training recipe.[3] Direct casting a trained FP32 model to MXFP4 for both weights and activations often lost substantial accuracy. ResNet-50 ImageNet top-1 fell from 77.40 to 42.39, and MobileNet v2 fell from 72.14 to 0.25. Fine-tuning recovered much of this, bringing ResNet-50 back to 74.86.[3] Mixed configurations held up better. GPT3-175B with MXFP4 weights and MXFP6 activations scored 0.754 on Lambada against an FP32 baseline of 0.755.[3] For training, GPT-style models from 20 million to 1.5 billion parameters were trained with MXFP4 weights and MXFP6 E3M2 activations and gradients, without changing the training recipe. At 1.5 billion parameters the final loss was 2.76, against 2.74 for FP32.[3]
Later work tried to put the matrix multiplications themselves in MXFP4:
| Work | Approach | Reported result |
|---|---|---|
| Tseng, Yu, Park, "Training LLMs with MXFP4" (arXiv:2502.20586, February 2025) | Uses MXFP4 GEMMs in the backward pass. Scales inputs by 3/4 before quantizing to avoid clipping, uses stochastic rounding for unbiased gradients, and applies a random Hadamard transform to bound the variance from block-level outliers | GPT models up to 6.7B parameters; gap to BF16 mixed precision under 0.1 validation perplexity; estimated backward-pass speedups above 1.3x over FP8 and 1.7x over BF16 |
| Castro, Panferov, Alistarh and others, "Quartet" (arXiv:2505.14669, May 2025) | All major linear-layer computations in MXFP4, with custom CUDA kernels for Blackwell | Llama-2-architecture models pretrained on C4; the authors report linear-layer speedups of almost 2x over FP8 on an RTX 5090 |
| NVIDIA, "Pretraining Large Language Models with NVFP4" (arXiv:2509.25149, September 2025) | Same recipe applied to MXFP4 and NVFP4, with a Hadamard size of 32 for MXFP4 | MXFP4 reached a worse loss than NVFP4 and needed 36% more tokens to match it |
Sources: Tseng et al.;[4] Castro et al.;[8] NVIDIA.[5] All results are reported by the respective authors.
These papers converge on a few techniques for 4-bit training: stochastic rounding for gradients, Hadamard rotations against outliers, and a careful choice of scale rounding. These are additions to the format; the OCP specification does not require any of them.[1][4][5]
## See also
- [Microscaling formats](https://aiwiki.ai/wiki/microscaling_formats)
- [NVFP4](https://aiwiki.ai/wiki/nvfp4)
- [FP4](https://aiwiki.ai/wiki/fp4)
- [FP8](https://aiwiki.ai/wiki/fp8)
- [Quantization-aware training](https://aiwiki.ai/wiki/quantization_aware_training)
- [Mixed-precision training](https://aiwiki.ai/wiki/mixed_precision_training)
## References
1. Rouhani, B. D., et al. (AMD, Arm, Intel, Meta, Microsoft, NVIDIA, Qualcomm). "OCP Microscaling Formats (MX) Specification, Version 1.0." Open Compute Project, September 2023. https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf
2. Open Compute Project Foundation. "AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm Standardize Next-Generation Narrow Precision Data Formats for AI." 17 October 2023. https://www.opencompute.org/blog/amd-arm-intel-meta-microsoft-nvidia-and-qualcomm-standardize-next-generation-narrow-precision-data-formats-for-ai
3. Rouhani, B. D., Zhao, R., More, A., et al. "Microscaling Data Formats for Deep Learning." arXiv:2310.10537, October 2023. https://arxiv.org/abs/2310.10537
4. Tseng, A., Yu, T., Park, Y. "Training LLMs with MXFP4." arXiv:2502.20586, February 2025. https://arxiv.org/abs/2502.20586
5. NVIDIA, Abecassis, F., et al. "Pretraining Large Language Models with NVFP4." arXiv:2509.25149, September 2025. https://arxiv.org/abs/2509.25149
6. Alvarez, E., Almog, O., Chung, E., Layton, S., Stosic, D., Krashinsky, R., Aubrey, K. "Introducing NVFP4 for Efficient and Accurate Low-Precision Inference." NVIDIA Technical Blog, 24 June 2025. https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/
7. Egiazarian, V., Castro, R. L., Kuznedelev, D., et al. "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization." arXiv:2509.23202, September 2025. https://arxiv.org/abs/2509.23202
8. Castro, R. L., Panferov, A., Tabesh, S., et al. "Quartet: Native FP4 Training Can Be Optimal for Large Language Models." arXiv:2505.14669, May 2025. https://arxiv.org/abs/2505.14669
9. OpenAI. "gpt-oss-120b & gpt-oss-20b Model Card." arXiv:2508.10925, August 2025. https://arxiv.org/abs/2508.10925
10. OpenAI. "gpt-oss" repository README (Reference Triton implementation; Precision format). GitHub. https://github.com/openai/gpt-oss
11. OpenAI. "openai/gpt-oss-120b" model card. Hugging Face. https://huggingface.co/openai/gpt-oss-120b
12. vLLM Team. "vLLM Now Supports gpt-oss." vLLM Blog, 5 August 2025. https://blog.vllm.ai/2025/08/05/gpt-oss.html
13. Gerganov, G. "llama : add gpt-oss." llama.cpp pull request #15091, merged 5 August 2025. https://github.com/ggml-org/llama.cpp/pull/15091
14. ggml-org. "ggml-common.h" and "ggml.h" (block_mxfp4, GGML_TYPE_MXFP4). llama.cpp source. Accessed 30 October 2026. https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/ggml-common.h
15. NVIDIA. "Parallel Thread Execution ISA" (Block Scaling for mma.sync; Tables 44-45). CUDA documentation. Accessed 30 October 2026. https://docs.nvidia.com/cuda/parallel-thread-execution/index.html
16. NVIDIA. "CUDA GPU Compute Capability." NVIDIA Developer. Accessed 30 October 2026. https://developer.nvidia.com/cuda-gpus
17. AMD. "AMD Instinct MI355X GPU" (specifications). Accessed 30 October 2026. https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html
18. AMD. "AMD Instinct MI455X GPU" (specifications). Accessed 30 October 2026. https://www.amd.com/en/products/accelerators/instinct/mi400/mi455x.html
19. AWS Neuron. "Trainium3 Architecture." AWS Neuron documentation. Accessed 30 October 2026. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/arch/neuron-hardware/trainium3.html
20. AWS Neuron. "Trainium3 Architecture Guide for NKI." AWS Neuron documentation. Accessed 30 October 2026. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/nki/guides/architecture/trainium3_arch.html
21. Liu, Y. "3 Ways to Boost MoE Inference on NVIDIA DGX Spark." NVIDIA Technical Blog, 27 October 2026. https://developer.nvidia.com/blog/3-ways-to-boost-moe-inference-on-nvidia-dgx-spark/
22. inclusionAI. "Ling-3.0-flash-fp4" model card and config.json. Hugging Face. Accessed 30 October 2026. https://huggingface.co/inclusionAI/Ling-3.0-flash-fp4
23. Microsoft. "microxcaling: MX PyTorch Emulation Library." GitHub. https://github.com/microsoft/microxcaling
24. PyTorch. "Quantized Inference" (torchao documentation; mxfp8, nvfp4 and mxfp4 sections). Accessed 30 October 2026. https://pytorch.org/ao/main/workflows/inference.html