AI Accelerator Comparison (H100 vs B200 vs MI300 vs TPU)

As of July 2026, the best AI accelerator depends on the metric you care about, so the honest answer to "H100 vs B200 vs MI300 vs TPU" is not a single winner. For raw single-GPU inference compute, NVIDIA's Blackwell Ultra B300 leads at 15 PFLOPS of dense FP4 ^[1]. At rack scale nothing matches NVIDIA's GB300 NVL72, which fuses 72 B300 GPUs into one 130 TB/s NVLink domain delivering about 1.1 exaFLOPS of dense FP4 ^[1]. AMD's Instinct MI355X ties the B300 for the most on-chip memory (288 GB HBM3e) and memory bandwidth (8 TB/s) and slightly leads it on dense BF16 and FP8, which makes it the value pick for memory-bound inference, but it trails the B300 on FP4 ^[2][1]. Google's TPU v7 Ironwood matches a single B200 on memory (192 GB) and bandwidth (7.4 TB/s) and scales to 9,216-chip pods, but is available only on Google Cloud ^[3]. The older H100 and H200 now lead only on price.

Which accelerator wins on each metric (July 2026)?

• Most memory per chip (shipping): AMD MI355X and NVIDIA B300 tie at 288 GB HBM3e. AMD's MI400 will raise this to 432 GB HBM4 in 2026 ^[2][4].
• Highest memory bandwidth per chip (shipping): NVIDIA B200/B300 and AMD MI355X at 8 TB/s; TPU Ironwood at 7.4 TB/s ^[1][2][3].
• Most single-GPU FP4 (inference) compute: NVIDIA B300 at 15 PFLOPS dense, ahead of the MI355X at 10.1 PFLOPS dense ^[1][2].
• Most single-GPU BF16/FP8 (training) compute: roughly a tie, with the MI355X a hair ahead (2.52 PFLOPS dense BF16, 5.03 PFLOPS dense FP8) over the B300 (2.5 and 5.0) ^[1][2].
• Best scale-up fabric: NVIDIA NVLink 5 (1.8 TB/s per GPU, 72-GPU 130 TB/s domain), far ahead of AMD Infinity Fabric and TPU ICI at the single-node level ^[1].
• Largest single interconnect domain: Google TPU Ironwood, whose pods reach 9,216 chips on one ICI fabric ^[3].
• Cheapest FP8 inference capacity: NVIDIA H100/H200 on the secondary market ^[5].

Verdict: what is the best AI GPU in 2026?

• Best overall for frontier training and inference at scale: NVIDIA GB300 NVL72 (Blackwell Ultra). Its rack-scale NVLink domain and the CUDA plus TensorRT-LLM software moat make it the default for the largest models ^[1].
• Best raw single-GPU inference part: NVIDIA B300, which leads dense FP4 (15 PFLOPS) and pairs it with 288 GB of HBM3e ^[1].
• Best value and best for memory-bound inference: AMD Instinct MI355X (288 GB, 8 TB/s, dense FP8 on par with the B300, lower price) ^[2][9].
• Cheapest way to buy FP8 inference throughput: NVIDIA H100 and H200 on the used and cloud-rental market ^[5].
• Best if you are already on Google Cloud: TPU v7 Ironwood, which matches a B200 on memory and bandwidth and scales to 9,216-chip pods, but cannot be bought, only rented ^[3].

AI accelerator spec comparison table

All FLOPS figures below are DENSE (non-sparse) tensor or matrix throughput at the labeled precision, so vendors are directly comparable. This matters because vendors quote sparsity differently. NVIDIA's marketing prints the 2x-larger "with sparsity" number, so the dense values here are half of NVIDIA's headline figures ^[1][7]. AMD's popular MI355X headline of 5 / 10 / 20 PFLOPS (BF16 / FP8 / FP4) is likewise the with-sparsity figure, so the dense values are 2.5 / 5.0 / 10.1 PFLOPS ^[2]; the older MI300X and MI325X headline (1,307 TFLOPS FP16) is already dense ^[6]. Google's TPU numbers are dense ^[3]. FP4 exists only on NVIDIA Blackwell and AMD CDNA 4 and later; Hopper, CDNA 3, and pre-Ironwood TPUs have no FP4 unit. Last verified: July 2026.

Scroll sideways for more →

Notes: H200 and H100 share the same Hopper compute, so their FLOPS are identical; only memory differs ^[1][8]. MI325X shares the MI300X compute die, so their FLOPS are identical; only memory differs ^[6]. TPU v5p and Trillium have no dedicated FP8 unit (FP8 runs at the BF16 rate), so no separate FP8 figure is listed; Ironwood is the first TPU with native FP8 ^[3]. The B200 also ships in a higher-clocked GB200 configuration (2.5 / 5.0 / 10 PFLOPS dense BF16 / FP8 / FP4); the row above lists the standalone HGX B200 ^[1]. † AMD's announced MI400 figures (40 PFLOPS FP4, 20 PFLOPS FP8) do not state a dense-vs-sparsity basis; if they follow AMD's MI355X convention they are with-sparsity, implying roughly half those values dense ^[4]. Google does not publish TPU process nodes; Ironwood is reported by third parties as TSMC N3 ^[3].

Rack-scale and pod-scale systems

Single-GPU numbers understate NVIDIA's real advantage, which is the rack. The table below aggregates full systems (dense FLOPS unless noted). Last verified: July 2026.

Scroll sideways for more →

AMD's answer at rack scale is the "Helios" MI400 rack, announced for 2026, which targets a 72-GPU scale-up domain at 260 TB/s to close AMD's biggest structural gap versus NVLink ^[4].

Which accelerator has the most memory and bandwidth?

Memory capacity and bandwidth are the primary bottleneck for large language model inference, where each generated token requires re-reading model weights and the KV cache. The AMD MI355X and NVIDIA B300 lead shipping parts at 288 GB HBM3e ^[2][1]. AMD held the capacity crown for most of 2023 to 2025: the MI300X shipped 192 GB when the H100 had only 80 GB, and the MI325X pushed to 256 GB ^[6]. NVIDIA matched AMD with the B300 at 288 GB and will be leapfrogged again in 2026 by the MI400 at 432 GB of HBM4 ^[4]. On bandwidth, B200, B300, and MI355X tie at 8 TB/s, ahead of Ironwood (7.4 TB/s), MI325X (6.0 TB/s), MI300X (5.3 TB/s), and H200 (4.8 TB/s) ^[1][2][3]. The H100's 3.35 TB/s now trails the field by more than 2x, which is why it is increasingly a training-throughput and cheap-inference part rather than a memory-bound-inference leader.

Which accelerator has the most compute, and how should you read the FLOPS?

The single most misread spec in this category is peak FLOPS, because vendors quote different precisions and different sparsity conventions. Three rules make the numbers comparable:

1. Dense vs sparsity. NVIDIA's datasheets print the "with 2:4 structured sparsity" number, which is exactly 2x the dense number, marked with an asterisk ^[1][7]. AMD's datasheets list both a dense and a "with structured sparsity" value, and AMD's popular MI355X marketing figures (5 / 10 / 20 PFLOPS) are the with-sparsity ones, so the dense equivalents are 2.5 / 5.0 / 10.1 PFLOPS ^[2]. Most production inference does not use 2:4 sparsity, so the dense figures above are the ones that matter. Google's TPU numbers are dense.
2. Precision matters. A B200 does 2.25 PFLOPS dense BF16 but 9 PFLOPS dense FP4, a 4x spread. Always compare the same precision. FP4 is inference-oriented; BF16 and FP8 are the training-relevant numbers.
3. Chip vs rack. A single B200 does 9 PFLOPS dense FP4; a GB200 NVL72 rack of 72 of them does 720 PFLOPS dense FP4. Do not compare a rack figure to a single-chip figure.

On single-GPU dense compute, NVIDIA and AMD trade blows by precision. The MI355X edges ahead on BF16 (2.52 versus 2.5 PFLOPS) and FP8 (5.03 versus 5.0 PFLOPS), while the B300 leads decisively on FP4 (15 versus 10.1 PFLOPS), the precision that matters most for inference ^[1][2]. Neither is a runaway on a single chip, and their memory (both 288 GB) and bandwidth (both 8 TB/s) are tied. NVIDIA's real separation shows up two ways. First, delivered performance: in MLPerf and independent SemiAnalysis testing, mature FP4 kernels in TensorRT-LLM plus 2:4 sparsity let the B300 open roughly a 1.3x delivered FP4 lead over the MI355X, wider than the paper gap ^[11]. Second, scale: NVIDIA sells the rack, not the chip, and no AMD or Google fabric currently matches a 72-GPU NVLink domain. Google's Ironwood sits between the two camps: its 4.61 PFLOPS dense FP8 per chip is close to a single B200, and it scales to 9,216-chip pods, but it is Google Cloud only ^[3].

Which has the best interconnect?

Interconnect is where NVIDIA's lead is widest. NVLink 5 gives each Blackwell GPU 1.8 TB/s and lets 72 GPUs act as one accelerator over a 130 TB/s non-blocking fabric ^[1]. AMD's Infinity Fabric tops out at node scale: the MI300X and MI325X expose 896 GB/s per GPU across seven links inside an eight-GPU server, the MI355X raises this to about 1,075 GB/s per GPU, but there is no rack-scale scale-up fabric until the MI400 Helios rack in 2026 ^[2][4][6]. Google takes a third path: modest per-chip ICI bandwidth (1.2 TB/s on Ironwood) but an enormous 3D-torus domain of up to 9,216 chips, which is how Google trains and serves Gemini without any NVLink-class fabric ^[3]. For a model that fits in one rack, NVLink wins; for a model sharded across thousands of chips, the TPU pod is a different and competitive shape.

Which process node and which vendor is newest?

The H100 and H200 use TSMC 4N; the B200 and B300 use the refined 4NP (N4P) ^[1][8]. AMD moved fastest down the node ladder: MI300X and MI325X compute dies are TSMC N5 (with N6 I/O dies), while the CDNA 4 MI355X moved to N3 (with N6 I/O), and the 2026 MI400 is expected on a 2 nm to 3 nm-class node that AMD has not officially confirmed ^[2][4][6]. Google does not disclose TPU nodes, though Ironwood is widely reported as N3 ^[3]. Newer nodes plus HBM generation, not raw transistor count, explain most of the bandwidth and efficiency gaps: HBM3e at 8 TB/s (B200, B300, MI355X) versus HBM3 at 3.35 to 5.3 TB/s (H100, MI300X) is the difference that decides most inference workloads.

What about performance per dollar?

Precise dollar-per-FLOP figures move weekly, but the rough 2026 picture is clear. NVIDIA H100 is the cheapest capacity: a used eight-GPU H100 HGX server traded for roughly 150,000 to 180,000 USD in early 2026, versus about 500,000 USD for a new B300 server, making H100 the lowest-cost path to dense FP8 inference throughput per dollar of capex ^[5]. AMD competes hardest on memory per dollar and cloud rental: MI325X rented for roughly 2.00 to 2.25 USD per GPU-hour in 2025, against 3.72 to 10.60 USD per GPU-hour for an H200, and its larger memory lets one card serve models that need two H200s or B200s ^[6][9]. NVIDIA's GB200 and GB300 NVL72 racks carry the highest capex, priced at roughly 3x an equivalent-GPU-count H100 server, but deliver 5x to 30x the inference throughput on frontier models, so their cost per token at high utilization is competitive or better ^[10]. Google TPUs cannot be bought at all; on Google Cloud they are frequently the cheapest cost per token for workloads already on the platform ^[3]. The rule of thumb: buy H100/H200 for cheap capacity, MI355X for memory-bound inference, Blackwell racks for frontier training and lowest cost per token at scale, and TPUs if you live on Google Cloud.

References

1. NVIDIA, "Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era" (developer blog, 2025), plus the NVIDIA HGX B200, GB200 NVL72, and GB300 NVL72 product pages (B300 dense NVFP4 15 PFLOPS, dense FP8 5 PFLOPS; B200 dense FP4 9, FP8 4.5, BF16 2.25 PFLOPS; NVLink 5 at 1.8 TB/s and 130 TB/s). https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/ and https://www.nvidia.com/en-us/data-center/hgx/ ↩
2. AMD, "AMD Instinct MI355X GPU" datasheet (288 GB HBM3E, 8 TB/s; dense BF16 2.5166 PFLOPS, dense FP8 5.0332 PFLOPS, dense FP4/FP6 10.0663 PFLOPS; with-sparsity values are 2x). https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/product-briefs/amd-instinct-mi355x-gpu-brochure.pdf ↩
3. Google Cloud, "Ironwood: The first Google TPU for the age of inference" (Apr 2025) and Cloud TPU documentation (tpu7x, v6e, v5p): Ironwood 192 GB HBM3E, 7.37 TB/s, dense FP8 4,614 TFLOPS, BF16 2,307 TFLOPS, 9,216-chip pods. https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/ironwood-tpu-age-of-inference/ and https://docs.cloud.google.com/tpu/docs/tpu7x ↩
4. AMD, "AMD Instinct MI350 Series and Beyond" blog and Advancing AI 2025 press release (MI400: up to 432 GB HBM4, 19.6 TB/s, 40 PFLOPS FP4 / 20 PFLOPS FP8 announced; Helios 72-GPU rack at 260 TB/s). https://www.amd.com/en/blogs/2025/amd-instinct-mi350-series-and-beyond-accelerating-the-future-of-ai-and-hpc.html ↩
5. NVIDIA H100 and secondary-market pricing, aggregated cloud and reseller listings (2026), reported by Tom's Hardware and The Next Platform. https://www.tomshardware.com/ ↩
6. AMD, "AMD Instinct MI300X" and "MI325X" datasheets (dense FP16 1,307.4 TFLOPS, dense FP8 2,614.9 TFLOPS; 192 GB HBM3 at 5.3 TB/s and 256 GB HBM3E at 6.0 TB/s; 7x128 GB/s Infinity Fabric). https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300x-data-sheet.pdf ↩
7. NVIDIA, "NVIDIA H100 Tensor Core GPU Datasheet" (FP16/BF16 1,979 TFLOPS and FP8 3,958 TFLOPS both marked "with sparsity," so dense is 989.5 and 1,979 TFLOPS). https://resources.nvidia.com/en-us-gpu-resources/h100-datasheet-24306 and https://www.nvidia.com/en-us/data-center/h100/ ↩
8. NVIDIA, "NVIDIA H200 Tensor Core GPU" product page (141 GB HBM3e, 4.8 TB/s, same Hopper compute as H100). https://www.nvidia.com/en-us/data-center/h200/ ↩
9. Cloud GPU rental price surveys, 2025 (MI325X vs H200 per-GPU-hour). https://www.thenextplatform.com/ ↩
10. NVIDIA GB200 NVL72 product page and pricing analysis. https://www.nvidia.com/en-us/data-center/gb200-nvl72/ ↩
11. SemiAnalysis, InferenceX and Blackwell versus MI355X delivered-performance and TCO analyses (2025). https://www.semianalysis.com/ ↩