# FLOPS

> Source: https://aiwiki.ai/wiki/flops
> Updated: 2026-10-15
> Categories: AI Compute, AI Hardware, Machine Learning
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "FLOPS." aiwiki.ai, 15 Oct 2026. https://aiwiki.ai/wiki/flops
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

FLOPS (floating-point operations per second, also written FLOP/s) is a measure of computing speed: the number of arithmetic operations on floating-point numbers that a processor, accelerator or supercomputer performs, or can theoretically perform, in one second. It is a rate, and it is often confused with FLOP (frequently pluralized as FLOPs), a count of operations; the compute used to train an AI model is a total measured in FLOP, so a system sustaining 10^18 FLOPS for one day performs 8.64 × 10^22 FLOP [1][2]. FLOPS ratings are the usual basis for comparing [GPUs](https://aiwiki.ai/wiki/gpu) and other AI accelerators and for ranking supercomputers, while FLOP counts express [training compute](https://aiwiki.ai/wiki/training_compute) budgets and regulatory thresholds. One chip's rating can vary more than a hundredfold with the number format and with whether structured sparsity is assumed, and a theoretical peak is not the speed real software reaches [3].

## What is the difference between FLOPS and FLOP?

A floating-point operation is one arithmetic step, such as an addition or a multiplication, on numbers stored in floating-point form. FLOPS counts those steps per second, so one teraFLOPS (TFLOPS) is 10^12 operations each second, whereas FLOP is a plain quantity. The abbreviations collide because "FLOPS" looks like the plural of FLOP and because "FLOPs" has been used for both the rate and the count. In a 2023 note, Lennart Heim, a co-author of [Epoch AI](https://aiwiki.ai/wiki/epoch_ai)'s method for estimating training compute, recommended writing FLOP for quantities and FLOP/s for performance, the convention that method uses [1][2]. Primary sources remain mixed: Meta's Llama 3 paper gives its flagship model's training budget as "3.8 × 10^25 FLOPs", a count, while NVIDIA's datasheets rate accelerators in TFLOPS and PFLOPS, which are rates [4][5].

Counts are also written as a rate multiplied by a duration. Jared Kaplan and colleagues' 2020 paper on [scaling laws](https://aiwiki.ai/wiki/scaling_laws) and OpenAI's [GPT-3](https://aiwiki.ai/wiki/gpt-3) paper report compute in petaflop/s-days, where one PF-day is 10^15 × 24 × 3,600 = 8.64 × 10^19 operations; by that measure GPT-3 175B took 3,640 PF-days, or 3.14 × 10^23 FLOP [6][7]. Policy texts use both units: the US [Executive Order 14110](https://aiwiki.ai/wiki/executive_order_14110) of October 2023 set a count for models (10^26 operations) and a rate for computing clusters (10^20 operations per second) [8].

### How are operations counted and peaks calculated?

Matrix hardware executes fused multiply-add (FMA) instructions, and by convention each FMA counts as two floating-point operations. NVIDIA's A100 whitepaper describes each streaming multiprocessor as delivering 1,024 FP16 FMA operations per clock, "or 2048 individual FP16 floating point operations per clock", and Kaplan's team attributes the factor of two in its per-token compute estimate to the same multiply-accumulate step [9][6]. Integer throughput, such as INT8, is quoted in operations per second (TOPS or POPS) rather than FLOPS [5].

A vendor's peak rating is arithmetic, not measurement: execution units, times operations per unit per clock cycle, times clock frequency. The [NVIDIA A100](https://aiwiki.ai/wiki/nvidia_a100) has 108 streaming multiprocessors, each completing 2,048 FP16 operations per clock on its [Tensor Cores](https://aiwiki.ai/wiki/tensor_core), and a boost clock of 1,410 MHz. Multiplying 108 × 2,048 × 1.41 GHz gives about 312 TFLOPS, NVIDIA's dense FP16 Tensor Core rating, and NVIDIA states that its peak rates are "based on GPU Boost Clock" [9]. Whether software keeps the units busy is a separate question.

## Prefixes and scale

FLOPS figures use the decimal SI prefixes, each step 1,000 times the previous one. Supercomputers passed one exaFLOPS on the 64-bit Linpack benchmark in 2022, and suppliers of [AI compute](https://aiwiki.ai/wiki/ai_compute) now describe low-precision systems in exaFLOPS and zettaFLOPS [10][11][12].

| Prefix | Symbol | Operations per second | Example |
|---|---|---|---|
| gigaFLOPS | GFLOPS | 10^9 | Green500 efficiency rankings use FP64 GFLOPS per watt; the June 2026 leader, KAIROS at the University of Toulouse, reached 73.28 GFLOPS/W [13] |
| teraFLOPS | TFLOPS | 10^12 | NVIDIA H100 SXM: 33.5 TFLOPS of FP64 vector math and 989.4 TFLOPS of dense BF16 tensor math [3] |
| petaFLOPS | PFLOPS | 10^15 | NVIDIA B200 (HGX): 2.25 PFLOPS of dense BF16 and 9 PFLOPS of dense FP4 [5] |
| exaFLOPS | EFLOPS | 10^18 | Frontier, the first system above 1 EFLOPS on FP64 Linpack (1.102 EFLOPS, June 2022) [10]; LineShine, 2.198 EFLOPS (June 2026) [14]; one 72-GPU GB300 NVL72 rack, 1,080 PFLOPS of dense FP4 [15] |
| zettaFLOPS | ZFLOPS | 10^21 | No FP64 system. Oracle OCI Zettascale10 (October 2025): "up to 16 zettaFLOPS of peak performance" across up to 800,000 NVIDIA GPUs, an FP4 figure [11][16]; NVIDIA Vera Rubin "AI factory" of 40,000 GPUs and 100 MW: 2 ZFLOPS of sparse NVFP4 [12] |

The lower rows mix precisions, so a prefix on its own says little: the GB300 NVL72 rack that reaches an exaFLOPS in dense FP4 is rated at 1.3 TFLOPS of FP64 per GPU, roughly 94 TFLOPS across its 72 GPUs [15].

## Why does numerical precision matter?

Every FLOPS figure refers to a particular number format. Narrower formats use fewer bits per value, so the same silicon completes more operations per cycle. On the [NVIDIA H100](https://aiwiki.ai/wiki/nvidia_h100) SXM, the dense tensor rating doubles from 989.4 TFLOPS in 16-bit formats to 1,978.9 TFLOPS in FP8, while the FP64 vector rate is 33.5 TFLOPS [3]. Epoch AI's 2023 survey of machine-learning hardware estimated that tensor-FP16 formats give roughly a tenfold speedup over FP32, and that INT8 on the H100 runs about 30 times faster than FP32 [17]. Doing most arithmetic in a narrow format while keeping sensitive values in a wider one is the basis of [mixed-precision training](https://aiwiki.ai/wiki/mixed_precision_training).

### Number formats

| Format | Bits | Layout | H100 SXM dense peak |
|---|---|---|---|
| FP64 | 64 | IEEE 754 double precision | 33.5 TFLOPS; 66.9 TFLOPS on Tensor Cores [3] |
| FP32 | 32 | IEEE 754 single precision | 66.9 TFLOPS [3] |
| TF32 | 19, computed from FP32 inputs | 1 sign, 8 exponent, 10 mantissa bits [9] | 494.7 TFLOPS [3] |
| BF16 | 16 | 1 sign, 8 exponent, 7 mantissa bits [9] | 989.4 TFLOPS [3] |
| FP16 | 16 | IEEE 754 half precision | 989.4 TFLOPS [3] |
| FP8 | 8 | E4M3 (4 exponent, 3 mantissa bits) or E5M2 (5 exponent, 2 mantissa bits) [3] | 1,978.9 TFLOPS [3] |
| FP6 | 6 | E3M2 or E2M3, used in MXFP6 blocks [18] | not supported [3] |
| FP4 | 4 | E2M1, used in MXFP4 and NVFP4 blocks [18][19] | not supported [3] |

TF32, introduced with the A100 in 2020, reads FP32 data but computes with FP16's 10-bit mantissa and FP32's exponent range, which [bfloat16](https://aiwiki.ai/wiki/bfloat16) also keeps [9]. [FP8](https://aiwiki.ai/wiki/fp8) reached NVIDIA hardware with the H100; its E4M3 encoding favors precision and E5M2 favors range [3][20].

Below eight bits, values are stored in small blocks that share a scale factor. The Open Compute Project's Microscaling (MX) specification version 1.0, published in 2023 by engineers from AMD, Arm, Intel, Meta, Microsoft, NVIDIA and Qualcomm, defines MXFP8, MXFP6, MXFP4 and MXINT8, each grouping 32 elements under one 8-bit power-of-two scale (E8M0) [18]. NVIDIA's [NVFP4](https://aiwiki.ai/wiki/nvfp4), introduced with Blackwell, uses 16-value blocks with an FP8 (E4M3) scale plus a per-tensor FP32 scale; NVIDIA says it delivers accuracy close to FP8, often within about 1% [19]. A 4-bit rating therefore covers 4-bit elements plus their share of a scale, and MXFP4 and NVFP4 ratings do not describe the same format (see [FP4](https://aiwiki.ai/wiki/fp4)).

### Comparing accelerators at a fixed precision

Because ratings move so much with format, accelerators can only be compared at a stated precision. The table gives dense (non-sparse) peak rates per chip from vendor documents, in TFLOPS. FP8 is the narrowest format with published rates for all seven chips, although Google says FP8 on the [TPU v5p](https://aiwiki.ai/wiki/google_tpu_v5p) is emulated at the BF16 rate, while [Ironwood](https://aiwiki.ai/wiki/tpu_ironwood), its seventh-generation [TPU](https://aiwiki.ai/wiki/tpu), supports it natively [21][22].

| Accelerator | Dense BF16 | Dense FP8 | Dense FP4 | FP64 |
|---|---|---|---|---|
| NVIDIA H100 SXM | 989.4 [3] | 1,978.9 [3] | not supported [3] | 33.5 (66.9 Tensor Core) [3] |
| NVIDIA B200 (HGX) | 2,250 [5] | 4,500 [5] | 9,000 [5] | 37 [5] |
| NVIDIA B300 (HGX) | 2,250 [15] | 4,500 [15] | 14,000 [15] | 1.2 [15] |
| AMD Instinct MI300X | 1,307.4 [23] | 2,614.9 [23] | not supported [23] | 81.7 (163.4 matrix) [23] |
| AMD Instinct MI355X | 2,516.6 [24] | 5,033.2 [24] | 10,066.3 (MXFP4) [24] | 78.6 [24] |
| Google TPU v5p | 459 [22] | 459 (emulated) [22][21] | not published [22] | not published [22] |
| Google TPU7x (Ironwood) | 2,307 [22] | 4,614 [22] | not published [22] | not published [22] |

NVIDIA's Blackwell datasheets quote sparse figures and state that dense is one-half; the [B200](https://aiwiki.ai/wiki/nvidia_b200) and [B300](https://aiwiki.ai/wiki/nvidia_b300) values above are those halves, except B300 FP4, which NVIDIA lists as 14 PFLOPS dense per GPU (its eight-GPU board total of 108 PFLOPS works out to 13.5) [5][15]. The HGX rows are eight-GPU boards configurable to 1,000 W per GPU (B200) and 1,100 W (B300); the corresponding GPUs in NVIDIA's 72-GPU GB200 and GB300 NVL72 racks are configurable to 1,200 W and 1,400 W and are rated higher, for example 2.5 PFLOPS of dense BF16 for a GB200 GPU [5][15].

Designers trade one precision against another. The Blackwell Ultra B300 raises dense FP4 from the B200's 9 PFLOPS to 14 PFLOPS but cuts FP64 from 37 TFLOPS to 1.2 TFLOPS [5][15]; NVIDIA describes Blackwell Ultra's dense NVFP4 throughput as 1.5 times Blackwell's [19]. AMD's [Instinct MI355X](https://aiwiki.ai/wiki/amd_instinct_mi355x) lists 78.6 TFLOPS of FP64 matrix throughput, about half the 163.4 TFLOPS of the [Instinct MI300X](https://aiwiki.ai/wiki/amd_instinct_mi300x), while nearly doubling its BF16 and FP8 rates [24][23].

Vendor comparisons can turn on the choice of format. AMD's MI355X product page claims up to 2.2 times the AI performance of "competitive accelerators" and charts the MI355X against NVIDIA's B200. Of the chart's three comparisons, only FP6 reaches that multiple: the MI355X runs FP6 at its FP4 rate (10.1 PFLOPS) and the B200 at its FP8 rate (4.5 PFLOPS dense), while the same chart gives 5.0 against 4.5 PFLOPS for FP16/BF16 and 10.1 against 9 PFLOPS for FP8, both with sparsity [25][5].

## Dense and sparse figures

Since the Ampere generation, NVIDIA's tensor cores have supported fine-grained structured [sparsity](https://aiwiki.ai/wiki/sparsity). Trained weights are pruned so that two of every four values are zero (the 2:4 pattern), the remaining weights are fine-tuned, and sparse tensor cores skip the zeros, which NVIDIA says doubles math throughput on those matrices [9]. The doubled rate applies only to matrix multiplications whose weights have been pruned to the pattern; a model trained and served densely runs at the dense rate.

Spec sheets often lead with the sparse number. NVIDIA's H100 product page lists 1,979 teraFLOPS for BF16 with an asterisk meaning "with sparsity", while the H100 whitepaper gives the dense value, 989.4 TFLOPS [20][3]. The Blackwell datasheet quotes most per-GPU rates in sparse form with the footnote "Dense is one-half of the sparse spec shown" [5]. AMD's MI300X and MI355X documents print dense and "with sparsity" columns side by side [23][24], and Google's TPU documentation lists a single peak per format [22].

The factor of two no longer holds everywhere. For the HGX B300, NVIDIA lists FP4 at 18 PFLOPS sparse and 14 PFLOPS dense per GPU [15]. NVIDIA's 2026 Vera Rubin datasheet labels 3,600 PFLOPS per NVL72 rack as a sparse "NVFP4 Inference" figure and 2,520 PFLOPS as a dense "NVFP4 Training" figure, a ratio of about 1.4 [12]. AMD gives no sparse rate at all for MXFP4 and MXFP6 on the MI355X [24].

Footnotes and asterisks say which figure is which, and where a document gives two numbers for one format with one exactly double the other, the larger is the sparse figure. Board and rack totals must be divided by the right GPU count: NVIDIA rates the HGX B200 board at 144 PFLOPS sparse and 72 PFLOPS dense in FP4, or 18 and 9 PFLOPS per GPU [5]. A comparison holds only when both sides use the same precision and the same sparsity assumption.

## Peak versus achieved performance

A rated peak assumes every execution unit is busy on every cycle, and real workloads fall short of it. Even on the dense linear algebra of the Linpack benchmark, LineShine, the June 2026 leader of the TOP500 list, measured about 80% of its theoretical peak [14]. Training large models typically reaches much less.

Google's 2022 [PaLM](https://aiwiki.ai/wiki/palm) paper introduced model FLOPs utilization (MFU): observed training throughput in tokens per second divided by the theoretical maximum throughput of the system at peak FLOPS, where the maximum counts only the operations needed for the forward and backward passes. The older hardware FLOPs utilization (HFU) counts everything the hardware executes, including rematerialization, in which forward-pass activations are recomputed to save memory; the authors argued that HFU depends on implementation choices and is not a consistent measure [26]. PaLM 540B, trained on 6,144 TPU v4 chips, reached 46.2% MFU and 57.8% HFU, with MFU computed against a peak of 275 TFLOPS per chip [26].

| Training run | Hardware | Utilization | Reported by |
|---|---|---|---|
| GPT-3 175B | NVIDIA V100 | 21.3% MFU | Google (PaLM paper), April 2022 [26] |
| Gopher 280B | 4,096 TPU v3 | 32.5% MFU | Google (PaLM paper), April 2022 [26] |
| Megatron-Turing NLG 530B | 2,240 NVIDIA A100 | 30.2% MFU | Google (PaLM paper), April 2022 [26] |
| PaLM 540B | 6,144 TPU v4 | 46.2% MFU, 57.8% HFU | Google, April 2022 [26] |
| Llama 3 405B | 8,192-16,384 NVIDIA H100 | 38-43% BF16 MFU (380-430 TFLOPS per GPU) | Meta, July 2024 [4] |
| Pangu Ultra 135B | 8,192 Huawei Ascend 910B | 52% MFU | Huawei, April 2025, as recorded by Epoch AI [27] |

Meta labels its figure "BF16 MFU", and the label matters. Its best configuration delivered 430 TFLOPS per GPU, 43% of the H100's dense BF16 peak of 989.4 TFLOPS; the same throughput would be about 22% of the dense FP8 peak and 11% of the sparse FP8 rating [4][3]. A utilization figure without its reference precision is ambiguous. Meta attributed the dip from 43% to 41% when moving from 8,192 to 16,384 GPUs to the smaller batch per data-parallel group needed to keep the global tokens per batch constant [4].

## How is training compute counted?

The standard shortcut comes from Kaplan and colleagues' January 2020 paper. A forward pass through a transformer with N non-embedding parameters costs about 2N FLOP per token and the backward pass about twice that, so training costs roughly 6N FLOP per token, or C ≈ 6ND for D training tokens [6]. The estimate omits attention terms that grow with context length, which the authors left out for models whose width is large relative to the context, and the GPT-3 paper likewise ignored attention as typically under 10% of compute [6][7]. By the same accounting, generating one token at inference, which needs only a forward pass, costs about 2N FLOP [6].

The rule reproduces published totals closely. GPT-3 175B had 174.6 billion parameters and 300 billion training tokens, and 6 × 174.6 × 10^9 × 300 × 10^9 gives the 3.14 × 10^23 FLOP the paper reports [7]. For Meta's 405-billion-parameter model, released as [Llama 3.1](https://aiwiki.ai/wiki/llama_3_1) 405B, 6 × 405 × 10^9 × 15.6 × 10^12 ≈ 3.8 × 10^25, matching Meta's stated budget [4]. For mixture-of-experts models, N is the number of parameters active per token. [DeepSeek-V3](https://aiwiki.ai/wiki/deepseek_v3) has 671 billion parameters but activates 37 billion per token and was trained on 14.8 trillion tokens [28], so 6 × 37 × 10^9 × 14.8 × 10^12 ≈ 3.3 × 10^24 FLOP, the figure Epoch AI records [27].

When parameter and token counts are not disclosed, compute can be estimated from hardware: number of chips × peak FLOP/s × training time × utilization. Epoch AI's 2022 methodology suggested assuming 30% utilization for large language models and 40% for other networks [2]. Its estimate for OpenAI's GPT-6 Astra, released on 3 September 2026, starts from statements by OpenAI and its partners that the model was trained on at least 100,000 GB200 GPUs in Abilene, Texas, and assumes about 90 days at 25% FP8 MFU, giving a central figure of 9.4 × 10^26 FLOP with an interval of roughly 5 × 10^26 to 2 × 10^27 [27]. The arithmetic shows a rate turning into a count: 100,000 GPUs × 5 PFLOPS of dense FP8 per GB200 GPU [5] × 7.8 million seconds × 0.25 ≈ 9.7 × 10^26 FLOP.

Selected estimates from Epoch AI's database, which in its 15 October 2026 update lists four models above 10^26 FLOP, the first being xAI's Grok 3 in February 2025 [27]:

| Model | Developer | Released | Estimated training compute (FLOP) | Epoch confidence |
|---|---|---|---|---|
| GPT-3 175B | OpenAI | 2020-05-28 | 3.14 × 10^23 | Confident |
| GPT-4 | OpenAI | 2023-03-15 | 2.1 × 10^25 | Likely |
| Llama 3.1 405B | Meta | 2024-07-23 | 3.8 × 10^25 | Confident |
| Grok 3 | xAI | 2025-02-17 | 3.5 × 10^26 | Likely |
| GPT-4.5 | OpenAI | 2025-02-27 | 3.8 × 10^26 | Likely |
| Grok 4 | xAI | 2025-07-09 | 5 × 10^26 | Speculative |
| GPT-6 Astra | OpenAI | 2026-09-03 | 9.4 × 10^26 | Likely |

Epoch's February 2026 summary put the growth of training compute for frontier language models at about five times per year since 2020 [29].

## FLOP thresholds in AI policy

Several AI laws and executive orders use training compute as a trigger for obligations, a practice discussed under [compute governance](https://aiwiki.ai/wiki/compute_governance). Most such thresholds are counts of operations; the executive order's cluster test and the export-control metric below are rates.

| Measure | Jurisdiction | Threshold | Status (October 2026) |
|---|---|---|---|
| AI Act, Article 51(2) | European Union | More than 10^25 FLOP of cumulative training compute creates a presumption of high-impact capabilities (systemic risk) | In force; the Commission can amend the threshold by delegated act [30] |
| Executive Order 14110, section 4.2 | United States | More than 10^26 integer or floating-point operations (10^23 for models trained mainly on biological sequence data); clusters of 10^20 operations per second | Revoked on 20 January 2025 [8][31] |
| SB 53 (Transparency in Frontier Artificial Intelligence Act) | California | Foundation models trained with more than 10^26 operations, counting fine-tuning and later modifications | Signed 29 September 2025; in effect since 2026 [32][33] |
| RAISE Act, as amended | New York | Foundation models trained with more than 10^26 operations | Amendment signed 27 March 2026; effective 1 January 2027 [33] |
| ECCN 3A090 | United States (export controls) | Total processing performance of 4,800 or more, a rate-based measure | In force [34] |

The [EU AI Act](https://aiwiki.ai/wiki/eu_ai_act) sets its presumption at 10^25 FLOP, an order of magnitude below the US federal and state figures [30]. The Digital Omnibus on AI, which the European Commission proposed on 19 November 2025, mainly concerns deadlines for high-risk systems, enforcement and administrative simplification; a March 2026 summary by the European Parliament's research service of the proposal and of the Council and Parliament positions listed no change to the compute presumption [35].

Federal reporting ended when President Donald Trump revoked Executive Order 14110 on 20 January 2025 [31]; California and New York have since written the 10^26 figure into state law. California's [SB 53](https://aiwiki.ai/wiki/california_sb_53) places heavier duties on "large frontier developers" with annual gross revenue above $500 million [32]. New York's amended RAISE Act, signed by Governor Kathy Hochul, uses the same compute definition and drops the dollar-cost test of the original December 2025 law [33].

US [export controls](https://aiwiki.ai/wiki/export_controls) on AI chips use a rate instead of a count. Under Export Control Classification Number 3A090, total processing performance (TPP) is 2 × MacTOPS × the bit length of the operation, where MacTOPS is the chip's theoretical peak of trillions of multiply-accumulate operations per second; chips with a TPP of 4,800 or more, or 1,600 or more combined with a performance density of at least 5.92, are controlled [34]. The regulation notes that 2 × MacTOPS may correspond to the TOPS or FLOPS on a datasheet, and the bit-length factor makes an 8-bit operation count for a quarter as much as a 32-bit one [34].

## TOP500 Linpack versus "AI FLOPS"

The TOP500 list ranks supercomputers by High-Performance Linpack (HPL), which solves a dense system of linear equations by LU factorization with partial pivoting. The operation count is fixed at 2/3 n^3 + O(n^2) double-precision operations, and methods that compute in lower precision and refine the answer afterward are not allowed; each entry reports Rmax, the best measured rate, next to Rpeak, the theoretical peak [36]. Frontier at Oak Ridge National Laboratory was the first system above one exaFLOPS, with 1.102 EFLOPS on the June 2022 list [10]. On the June 2026 list, LineShine at the National Supercomputing Centre in Shenzhen took first place with 2.198 EFLOPS using CPUs only, making five systems at or above one exaFLOPS alongside El Capitan (1.809), Frontier (1.353), Aurora (1.012) and JUPITER Booster (1.000) [14][13].

A companion benchmark, HPL-MxP, keeps the problem but lets the factorization run in lower precision (its description suggests 16-bit) before an iterative solver restores 64-bit accuracy [37]. On it, El Capitan reached 16.7 EFLOPS in June 2026, against 1.809 EFLOPS on standard HPL [13].

Claims of exaFLOPS-class "AI supercomputers" usually refer to theoretical peaks in 8-bit or 4-bit formats, sometimes with sparsity, and are not comparable with TOP500 results. NVIDIA rates a single 72-GPU GB300 NVL72 rack at 1.1 exaFLOPS of dense FP4 [19]. When Google announced Ironwood in April 2025, it said a 9,216-chip pod delivers 42.5 exaFLOPS, more than 24 times the 1.7 exaFLOPS of El Capitan [21]. Google's figure is a theoretical FP8 peak, while El Capitan's is a measured FP64 Linpack result [21][38]. The Register estimated El Capitan's own FP8 peak at a little over 87 exaFLOPS dense, and Google told the publication that it did not have information on El Capitan's sustained FP8 performance [38]. Oracle's "up to 16 zettaFLOPS" for OCI Zettascale10 [11] was reached, according to Heise, by adding up 20 PFLOPS of FP4 per Blackwell GPU [16], which is NVIDIA's sparse FP4 rating for a GB200 GPU [5]. Heise estimated the same hardware at about 32 exaFLOPS of FP64 with perfect scaling and noted that real performance would depend on latency between systems [16].

## Performance per dollar and per watt

Epoch AI's November 2023 study of machine-learning hardware found that peak FLOP/s of GPUs doubled about every 2.3 years, FP32 FLOP/s per dollar every 2.1 years for machine-learning GPUs and every 2.5 years for general GPUs, and FLOP/s per watt every 3.0 years for machine-learning GPUs [17]. An October 2024 analysis of tensor-FP16 performance found the leading ML chips becoming about 40% more energy-efficient each year, with Meta's MTIA at up to 2.1 × 10^12 FLOP/s per watt and the H100 at up to 1.4 × 10^12 [39]. Epoch's summary updated in February 2026 estimated that AI chip performance per dollar had grown 49% a year since 2023, doubling about every 1.7 years, and that GPUs had become 34% more energy-efficient per year since 2008 [29]. The rates differ because the studies use different chip samples, periods and precisions.

Precision also changes the cost per FLOP of a single chip. In March 2024, NVIDIA chief executive Jensen Huang told CNBC that Blackwell GPUs would cost $30,000 to $40,000 each, and CNBC cited analyst estimates of $25,000 to $40,000 for the H100 [40]. Against dense ratings, that is about $25-40 per BF16 TFLOPS for the H100 and $13-18 per BF16 TFLOPS for the B200, but about $3.3-4.4 per FP4 TFLOPS for the B200 [40][3][5]. Such figures cover purchase price only, not power, networking or buildings.

Energy-efficiency figures depend on the same choices. The Green500 ranks supercomputers by measured FP64 Linpack performance per watt, and its June 2026 leader, KAIROS, reached 73.28 GFLOPS/W [13]. An H100's dense BF16 peak of 989.4 TFLOPS at its 700 W maximum thermal design power works out to about 1.4 TFLOPS/W [3][20], roughly 19 times higher, but one is a measured whole-system FP64 result and the other a chip-level 16-bit peak. Google states Ironwood's efficiency in "peak FP8 flops delivered per watt of thermal design power per chip package" and says it doubles the performance per watt of Trillium, its previous TPU generation [21].

## References

1. Lennart Heim. "FLOP for Quantity, FLOP/s for Performance." Lennart Heim (blog), 2023-04-14. https://blog.heim.xyz/flop-for-quantity-flop-s-for-performance/
2. Jaime Sevilla, Lennart Heim, Marius Hobbhahn, Tamay Besiroglu, Anson Ho and Pablo Villalobos. "Estimating Training Compute of Deep Learning Models." Epoch AI, 2022-01-20. https://epoch.ai/blog/estimating-training-compute
3. "NVIDIA H100 Tensor Core GPU Architecture" (whitepaper, version 1.04). NVIDIA, 2023-05-04. https://dam-cdn.nvd.orangelogic.com/AssetLink/705n6ur546g0uk43w0117r17n8042d73.pdf
4. Llama Team, AI @ Meta. "The Llama 3 Herd of Models." arXiv:2407.21783, 2024-07-31. https://arxiv.org/abs/2407.21783
5. "NVIDIA Blackwell" (datasheet). NVIDIA, 2025-10-28. https://dam-cdn.nvd.orangelogic.com/AssetLink/y441155802qub41q118b2852i557jem5.pdf
6. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu and Dario Amodei. "Scaling Laws for Neural Language Models." arXiv:2001.08361, 2020-01-23. https://arxiv.org/abs/2001.08361
7. Tom B. Brown et al. "Language Models are Few-Shot Learners." arXiv:2005.14165, 2020-05-28. https://arxiv.org/abs/2005.14165
8. "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence." The White House (archived), 2023-10-30. https://bidenwhitehouse.archives.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/
9. "NVIDIA A100 Tensor Core GPU Architecture" (whitepaper). NVIDIA, 2020-08-25. https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf
10. "ORNL's Frontier First to Break the Exaflop Ceiling." TOP500, 2022-05-30. https://www.top500.org/news/ornls-frontier-first-to-break-the-exaflop-ceiling/
11. "Oracle Unveils Next-Generation Oracle Cloud Infrastructure Zettascale10 Cluster for AI." Oracle, 2025-10-14. https://www.oracle.com/news/announcement/ai-world-oracle-unveils-next-generation-oci-zettascale10-cluster-for-ai-2025-10-14/
12. "NVIDIA Vera Rubin" (datasheet). NVIDIA, 2026-10-08. https://dam-cdn.nvd.orangelogic.com/AssetLink/56p68o47y6f1yucpubl0ump1c0aif422.pdf
13. "Highlights - June 2026." TOP500, accessed 2026-10-15. https://www.top500.org/lists/top500/2026/06/highs/
14. "LineShine Debuts at No. 1 as the TOP500 Enters a New Global Exascale Era." TOP500, 2026-06-23. https://top500.org/news/lineshine-debuts-no-1-top500-enters-new-global-exascale-era/
15. "NVIDIA Blackwell Ultra" (datasheet). NVIDIA, 2025-10-01. https://dam-cdn.nvd.orangelogic.com/AssetLink/1k0p832eq8r5ca0u5383ie5o4tp3bst1.pdf
16. Mark Mantel. "Zettascale10: Oracle announces fastest supercomputer cluster." heise online, 2025-10-15. https://heise.de/-10767366
17. Marius Hobbhahn, Lennart Heim and Gökçe Aydos. "Trends in Machine Learning Hardware." Epoch AI, 2023-11-09. https://epoch.ai/blog/trends-in-machine-learning-hardware
18. "OCP Microscaling Formats (MX) Specification, Version 1.0." Open Compute Project, 2023-10-13. https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf
19. "Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era." NVIDIA Technical Blog, 2025-08-22. https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/
20. "H100 GPU" (product page and specifications). NVIDIA, accessed 2026-10-15. https://www.nvidia.com/en-us/data-center/h100/
21. Amin Vahdat. "Ironwood: The first Google TPU for the age of inference." Google, 2025-04-09. https://blog.google/products/google-cloud/ironwood-tpu-age-of-inference/
22. "TPU7x (Ironwood)." Google Cloud Documentation, accessed 2026-10-15. https://docs.cloud.google.com/tpu/docs/tpu7x
23. "AMD Instinct MI300X Accelerator" (data sheet). AMD, 2025-06-16. https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300x-data-sheet.pdf
24. "AMD Instinct MI355X GPU" (brochure). AMD, 2025-06-16. https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/product-briefs/amd-instinct-mi355x-gpu-brochure.pdf
25. "AMD Instinct MI355X GPUs" (product page). AMD, accessed 2026-10-15. https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html
26. Aakanksha Chowdhery et al. "PaLM: Scaling Language Modeling with Pathways." arXiv:2204.02311, 2022-04-05. https://arxiv.org/abs/2204.02311
27. "Data on AI Models." Epoch AI, 2026-10-15. https://epoch.ai/data/ai-models
28. DeepSeek-AI. "DeepSeek-V3 Technical Report." arXiv:2412.19437, 2024-12-27. https://arxiv.org/abs/2412.19437
29. "Trends in AI." Epoch AI, updated 2026-02-05. https://epoch.ai/trends
30. "Article 51: Classification of General-Purpose AI Models as General-Purpose AI Models with Systemic Risk." EU Artificial Intelligence Act (Future of Life Institute), accessed 2026-10-15. https://artificialintelligenceact.eu/article/51/
31. "Initial Rescissions of Harmful Executive Orders and Actions." The White House, 2025-01-20. https://www.whitehouse.gov/presidential-actions/2025/01/initial-rescissions-of-harmful-executive-orders-and-actions/
32. Justine Gluck. "California's SB 53: The First Frontier AI Law, Explained." Future of Privacy Forum, 2025-10-03. https://fpf.org/blog/californias-sb-53-the-first-frontier-ai-law-explained/
33. "New York's Frontier AI Law Gets a California Makeover, With Some Key Differences." Cooley, 2026-03-31. https://www.cooley.com/news/insight/2026/2026-03-31-new-yorks-frontier-ai-law-gets-a-california-makeover-with-some-key-differences
34. "Supplement No. 1 to Part 774, The Commerce Control List" (ECCN 3A090 and technical notes). Electronic Code of Federal Regulations, Title 15, accessed 2026-10-15. https://www.ecfr.gov/current/title-15/subtitle-B/chapter-VII/subchapter-C/part-774/appendix-Supplement%20No.%201%20to%20Part%20774
35. Maria Niestadt. "Parliament's emerging position on the Digital Omnibus on AI." European Parliamentary Research Service, 2026-03-23. https://www.europarl.europa.eu/RegData/etudes/ATAG/2026/785685/EPRS_ATA(2026)785685_EN.pdf
36. "The Linpack Benchmark." TOP500, accessed 2026-10-15. https://top500.org/project/linpack/
37. "HPL-MxP Mixed-Precision Benchmark." HPL-MxP, accessed 2026-10-15. https://hpl-mxp.org/
38. Tobias Mann. "Google offers 7th-gen Ironwood TPUs for AI, with AI-inspired comparisons." The Register, 2025-04-10. https://www.theregister.com/2025/04/10/googles_7thgen_ironwood_tpus_debut/
39. Robi Rahman. "Leading ML hardware becomes 40% more energy-efficient each year." Epoch AI, 2024-10-23. https://epoch.ai/data-insights/ml-hardware-energy-efficiency
40. Kif Leswing. "Nvidia's Blackwell AI chip will cost more than $30,000, CEO says." CNBC, 2024-03-19. https://www.cnbc.com/2024/03/19/nvidias-blackwell-ai-chip-will-cost-more-than-30000-ceo-says.html