# Cloud TPU

> Source: https://aiwiki.ai/wiki/cloud_tpu
> Updated: 2026-10-15
> Fact-checked: 2026-10-15
> Categories: AI Compute, AI Hardware, AI Infrastructure, Machine Learning
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Cloud TPU." aiwiki.ai, 15 Oct 2026. https://aiwiki.ai/wiki/cloud_tpu
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

Cloud TPU is [Google Cloud](https://aiwiki.ai/wiki/google_cloud)'s offering of [Tensor Processing Units](https://aiwiki.ai/wiki/tpu) (TPUs), the family of custom [application-specific integrated circuits](https://aiwiki.ai/wiki/asic) (ASICs) that [Google](https://aiwiki.ai/wiki/google) builds to accelerate [machine learning](https://aiwiki.ai/wiki/machine_learning) training and inference. First deployed internally in 2015 and publicly announced in May 2016, TPUs are designed from the ground up for [neural network](https://aiwiki.ai/wiki/neural_network) computation rather than general-purpose processing, and Google rents them to external users through Google Cloud as Cloud TPUs, a service that opened to customers as a beta in February 2018.[20][21] As of October 2026 Google had built seven TPU generations; the newest generally available one is [TPU Ironwood](https://aiwiki.ai/wiki/tpu_ironwood) (TPU7x), whose 9,216-chip superpod delivers 42.5 exaFLOPS of FP8 compute, and the eighth generation, split into the TPU 8t and TPU 8i chips, was announced in April 2026 and listed as coming soon.[3][9][19] Its external customers include [Anthropic](https://aiwiki.ai/wiki/anthropic), which in October 2025 announced plans to use up to one million TPUs and in April 2026 signed for multiple gigawatts of next-generation TPU capacity.[15][17]

## ELI5 (Explain Like I'm 5)

Imagine your brain is really good at lots of different things: reading, drawing, playing games, and doing math. That is like a regular computer chip (a CPU or GPU). Now imagine a special calculator that can only do one kind of math problem, but it does that one problem incredibly fast. That is what a TPU is. Google built this special calculator because training an [AI model](https://aiwiki.ai/wiki/model) requires doing the same type of math (multiplying big grids of numbers) over and over, billions of times. By making a chip that only does this one job, Google made AI training and inference much faster and cheaper than using a regular chip that tries to do everything.

## What is a Cloud TPU?

A Cloud TPU is a Google-designed matrix-math accelerator that customers provision on demand through Google Cloud rather than buying physical hardware. Where a [GPU](https://aiwiki.ai/wiki/gpu) is a general-purpose parallel processor, a TPU is an ASIC whose silicon is dominated by a [systolic array](https://aiwiki.ai/wiki/systolic_array) of multiply-accumulate units optimized for the dense [matrix multiplication](https://aiwiki.ai/wiki/matrix_multiplication) at the heart of deep learning. Google's documentation lists Compute Engine, Google Kubernetes Engine (GKE) and Vertex AI as the services through which customers use TPUs.[23] For most of the product's history Google did not sell TPU chips, so Cloud TPU was both the brand name and the only external access channel for the hardware. That began to change in 2025 and 2026, when Google started placing TPUs in other providers' data centers and delivering TPU systems to customers' own facilities.[52][55]

## History and motivation

Google began developing TPUs around 2013 in response to an internal projection that if people used voice search for three minutes a day, the company would need to double its [data center](https://aiwiki.ai/wiki/data_center) compute capacity.[1] At the time, running [deep learning](https://aiwiki.ai/wiki/deep_learning) inference on CPUs and GPUs was expensive in terms of both cost and power consumption. Google engineers, including Norman Jouppi, the first author of the TPU's architecture paper, designed a purpose-built chip that could handle neural network inference at scale with far better performance per watt than existing hardware.[1]

The first TPU (v1) was deployed in Google data centers in 2015 and publicly disclosed at the Google I/O conference in May 2016.[21] Jouppi and colleagues published the landmark paper "In-Datacenter Performance Analysis of a Tensor Processing Unit" at the International Symposium on Computer Architecture (ISCA) in June 2017. The paper reported that the TPU was on average about 15 to 30 times faster than contemporary CPUs and GPUs on Google's production neural network inference workloads, with 30 to 80 times higher performance per watt (TOPS per watt) [1].

Google announced Cloud TPUs, based on TPU v2, in 2017 and opened them to Google Cloud customers as a beta in February 2018, when a Cloud TPU (a board of four TPU v2 chips with up to 180 teraFLOPS and 64 GB of HBM) cost $6.50 per hour, billed by the second.[8][20][48] Since then, each new generation has expanded capabilities from inference-only (v1) to both training and inference (v2 onward), while continuously scaling up in compute power, memory bandwidth, and interconnect speed.

## Architecture

### Systolic array design

At the core of every TPU is a systolic array, a grid of multiply-accumulate (MAC) units through which data flows in a rhythmic, pipelined fashion. In a systolic array, partial results move from one processing element to the next without returning to memory at each step. This design minimizes memory access overhead and maximizes throughput for matrix multiplication, which is the dominant operation in neural network training and inference.

The original TPU v1 contained a single 256 x 256 systolic array of 8-bit multiply-accumulate units, providing 65,536 MACs that could perform up to 92 trillion operations per second (TOPS).[1] Starting with TPU v2, the matrix units were 128 x 128 arrays operating on [bfloat16](https://aiwiki.ai/wiki/bfloat16) inputs with FP32 accumulation. TPU v6e and TPU7x (Ironwood) use larger 256 x 256 matrix multiply units (MXUs), increasing per-cycle throughput.[23]

### TensorCore

Starting from TPU v2, each [TPU chip](https://aiwiki.ai/wiki/tpu_chip) contains one or more TensorCores. A TensorCore is a self-contained compute unit that includes:

- **Matrix Multiply Units (MXUs):** The primary compute engines, each containing a systolic array of multiply-accumulators. [TPU v4](https://aiwiki.ai/wiki/google_tpu_v4), v5e and v5p TensorCores have four MXUs each; the TPU v6e TensorCore has two.[6][25][26][27]
- **Vector unit:** Handles element-wise operations such as [activation functions](https://aiwiki.ai/wiki/activation_function), [softmax](https://aiwiki.ai/wiki/softmax), and [batch normalization](https://aiwiki.ai/wiki/batch_normalization).
- **Scalar unit:** Manages control flow, memory address calculations, and other administrative operations.

TPU v2, v3, v4, v5p and TPU7x chips contain two TensorCores, while v5e and v6e chips contain one. Each v3 TensorCore has two 128 x 128 MXUs, v4, v5e and v5p TensorCores have four, and the v6e TensorCore has two 256 x 256 MXUs.[4][6][24][25][26][27][31] On TPU7x the two TensorCores sit on separate chiplets, and frameworks such as JAX expose each chip as two devices.[24]

### SparseCore

Starting with TPU v4, Google introduced SparseCores, specialized dataflow processors designed to accelerate models that rely heavily on sparse [embedding](https://aiwiki.ai/wiki/embeddings) lookups. Embedding-heavy models are common in [recommendation system](https://aiwiki.ai/wiki/recommendation_system)s and ranking workloads. TPU v4 includes four SparseCores per chip, each with dedicated scratchpad memory and optimized dataflow for sparse memory access patterns. Models with ultra-large embeddings have achieved 5 to 7 times speedups using SparseCores while consuming only about 5% of the total chip die area and power budget.[5] [TPU v5p](https://aiwiki.ai/wiki/google_tpu_v5p) features second-generation SparseCores, TPU v6e includes third-generation SparseCores (two per chip), and TPU v5p and TPU7x each have four SparseCores per chip.[7][8][23]

### The bfloat16 number format

Google developed the bfloat16 (Brain Floating Point) number format specifically for TPU-based machine learning workloads. Bfloat16 is a 16-bit floating-point representation that uses one sign bit, eight exponent bits, and seven mantissa bits. Unlike IEEE FP16 (which trades exponent range for precision), bfloat16 preserves the same dynamic range as FP32 while halving memory usage. This design choice reflects the observation that neural networks are more sensitive to dynamic range than to precision during training.

On Cloud TPUs, matrix multiplications are performed with bfloat16 inputs and accumulated in FP32, providing a practical balance between computational speed and numerical accuracy. Because bfloat16 multipliers are roughly half the silicon area of FP16 multipliers and eight times smaller than FP32 multipliers, TPUs can pack more compute into the same die area [2]. TPU7x adds native FP8 arithmetic, at twice its BF16 peak rate.[24]

### Memory and interconnects

TPU chips use [High Bandwidth Memory](https://aiwiki.ai/wiki/high_bandwidth_memory) (HBM) as their primary data store. HBM capacity and bandwidth have increased substantially with each generation, from 8 GB of DDR3 in TPU v1 to 192 GB of HBM3E per chip in TPU v7 (Ironwood).

TPU chips within a pod or slice communicate through high-speed Inter-Chip Interconnects (ICI). The network topology varies by generation:

- **2D torus:** TPU v2, v3, v5e, and v6e use a 2D torus topology, where each chip connects to its four nearest neighbors (north, south, east, west).
- **3D torus:** TPU v4 and v5p use a 3D torus topology, where each chip connects to six neighbors. This provides higher bisection bandwidth for large-scale [distributed training](https://aiwiki.ai/wiki/distributed_training). A full TPU v5p pod of 8,960 chips forms a 16 x 20 x 28 topology.[28]
- **TPU v7 (Ironwood):** Also a 3D torus, with 1.2 TB/s (9.6 Tb/s) of bidirectional ICI bandwidth per chip, enabling 9,216-chip superpods.[16][24]

## What are the TPU generations and their specs?

The following table summarizes the specifications of each TPU generation:

| Generation | Year | Process | Peak performance | HBM capacity | HBM bandwidth | Max pod size | Topology | Key feature |
|---|---|---|---|---|---|---|---|---|
| TPU v1 | 2015 | 28 nm | 92 TOPS (INT8) | 8 GB DDR3 | 34 GB/s | N/A (single board) | N/A | Inference only; 256x256 [systolic array](https://aiwiki.ai/wiki/systolic_array) |
| TPU v2 | 2017 | 16 nm | 45 TFLOPS (bf16) | 16 GB HBM | 600 GB/s | 256 chips (11.5 PFLOPS) | 2D torus | First TPU for training; introduced bfloat16 |
| TPU v3 | 2018 | 16 nm | 123 TFLOPS (bf16) | 32 GB HBM | 900 GB/s | 1,024 chips | 2D torus | Liquid cooling; 2.7x perf over v2 |
| [TPU v4](https://aiwiki.ai/wiki/google_tpu_v4) | 2021 | 7 nm | 275 TFLOPS (bf16) | 32 GB HBM | 1,200 GB/s | 4,096 chips | 3D torus | SparseCores; optical reconfigurable interconnect |
| [TPU v5e](https://aiwiki.ai/wiki/google_tpu_v5e) | 2023 | N/A | 197 TFLOPS (bf16) | 16 GB HBM | 819 GB/s | 256 chips | 2D torus | Cost-efficient; training and inference |
| [TPU v5p](https://aiwiki.ai/wiki/google_tpu_v5p) | 2023 | N/A | 459 TFLOPS (bf16) | 95 GB HBM | 2,765 GB/s | 8,960 chips | 3D torus | 2nd-gen SparseCores; 2.8x faster LLM training than v4 |
| [TPU v6e (Trillium)](https://aiwiki.ai/wiki/google_trillium) | 2024 | N/A | 918 TFLOPS (bf16) | 32 GB HBM | 1,640 GB/s | 256 chips | 2D torus | 4.7x perf over v5e; 3rd-gen SparseCores |
| [TPU v7 (Ironwood)](https://aiwiki.ai/wiki/tpu_ironwood) | 2025 | N/A | 4,614 TFLOPS (FP8); 2,307 TFLOPS (bf16) | 192 GB HBM3E | 7,370 GB/s | 9,216 chips | 3D torus (ICI 9.6 Tb/s) | Inference-optimized; 2x perf/watt over v6e |
| [TPU 8t](https://aiwiki.ai/wiki/google_tpu_8t) / [TPU 8i](https://aiwiki.ai/wiki/google_tpu_8i) | 2026 (announced) | Not disclosed | 12.6 / 10.1 PFLOPS (FP4) | 216 / 288 GB HBM | 6,528 / 8,601 GB/s | 9,600 (8t) / 1,152 (8i) chips | 3D torus (8t) / Boardfly (8i) | Separate training and inference chips; Axion hosts |

Figures for TPU v4 through TPU7x follow Google's documentation and announcements; the eighth-generation row uses Google's April 2026 technical description.[7][8][24][42]

### TPU v1

The first-generation TPU was designed exclusively for neural network inference. It featured a single 256 x 256 systolic array of 8-bit integer ALUs, 28 MiB of on-chip memory, and 8 GB of DDR3 memory. Operating at 700 MHz on a 28 nm process with a 75 W thermal design power, it drew 28 watts idle and 40 watts busy in Google's measurements while delivering a peak of 92 TOPS. TPU v1 was deployed as a coprocessor on the PCIe bus and was never offered as a standalone cloud product. It powered latency-sensitive Google services including Search ranking (RankBrain), Street View, Google Translate, Google Photos, and the [AlphaGo](https://aiwiki.ai/wiki/alphago) system in its 2016 matches against Lee Sedol [1][21][22].

### TPU v2

Announced at Google I/O in May 2017, TPU v2 was the first generation to support both training and inference. Each chip contained two TensorCores with 128 x 128 MXUs, 16 GB of HBM, and 600 GB/s memory bandwidth. TPU v2 introduced the bfloat16 number format and delivered 45 TFLOPS per chip. Pods of up to 256 chips provided 11.5 petaFLOPS of aggregate compute and 4 TB of HBM.[19][31] TPU v2 was the first generation made available to external users, as Cloud TPU boards of four chips.[20]

### TPU v3

Announced at Google I/O 2018, TPU v3 doubled the HBM capacity to 32 GB per chip and increased memory bandwidth to 900 GB/s. The clock speed rose from 700 MHz to 940 MHz, and peak performance reached 123 TFLOPS per chip. Pods scaled up to 1,024 chips, providing over 100 petaFLOPS of aggregate compute (126 petaFLOPS in bf16, according to Google's current documentation). TPU v3 was the first generation to require liquid cooling due to its higher power density [4][31][48].

### TPU v4

Announced at Google I/O 2021 and made generally available in 2022, TPU v4 represented a major architectural leap. Built on a 7 nm process with a die size under 400 mm2, it delivered 275 TFLOPS per chip. Each chip contained two TensorCores (four 128 x 128 MXUs each), four SparseCores, and 32 GB of HBM with 1,200 GB/s bandwidth.[5][27]

TPU v4 introduced a 3D torus interconnect topology with optically reconfigurable circuit switches (OCS), allowing dynamic reconfiguration of the network topology to match workload requirements. A full v4 pod contained 4,096 chips, and Google said at I/O 2021 that the pod's interconnect had ten times the bandwidth of competing systems.[32] Google described the TPU v4 pod as an "optically reconfigurable supercomputer" in a 2023 paper [5].

### TPU v5e

Announced in August 2023, [TPU v5e](https://aiwiki.ai/wiki/google_tpu_v5e) was designed as a cost-efficient accelerator for both training and inference. It delivers 197 TFLOPS in bfloat16 and 393 TOPS in INT8, with 16 GB of HBM per chip. Pods support up to 256 chips in a 2D torus topology [6]. Google called v5e its most cost-efficient TPU, with 2.3 times the price-performance of TPU v4, for workloads including [large language model](https://aiwiki.ai/wiki/large_language_model) fine-tuning and serving [7].

### TPU v5p

Announced in December 2023 alongside the [Gemini](https://aiwiki.ai/wiki/gemini) model, TPU v5p was Google's most powerful TPU at launch. Each chip delivers 459 TFLOPS in bfloat16 and 918 TOPS in INT8, with 95 GB of HBM and 2,765 GB/s bandwidth. A full v5p pod connects 8,960 chips in a 3D torus topology with 4,800 Gbps of ICI bandwidth per chip, which the JAX team's Scaling Book describes as a 16 x 20 x 28 arrangement [7][28]. TPU v5p features second-generation SparseCores that can train embedding-dense models 1.9x faster than TPU v4, and Google said it trains large LLMs 2.8x faster than TPU v4 [7].

### TPU v6e (Trillium)

Announced in May 2024 and made generally available in December 2024, [Trillium](https://aiwiki.ai/wiki/google_trillium) is Google's sixth-generation TPU. It achieves roughly 918 TFLOPS in bfloat16 per chip (approximately 4.7x the performance of TPU v5e) through larger 256 x 256 MXUs and a higher clock speed. HBM capacity doubled to 32 GB with doubled bandwidth (1,640 GB/s), and ICI bandwidth also doubled compared to v5e. Trillium includes third-generation SparseCores and is over 67% more energy efficient than TPU v5e.[8][19]

Trillium pods scale up to 256 chips, and with Multislice technology and Titanium IPUs (Intelligence Processing Units), multiple pods can be connected into building-scale supercomputers with tens of thousands of chips. Google reported up to 2.1x better performance per dollar than v5e and up to 2.5x better than v5p for training dense LLMs such as [Llama](https://aiwiki.ai/wiki/llama) 2-70B and Llama 3.1-405B [8][19].

### TPU v7 (Ironwood)

Ironwood is Google's seventh-generation TPU and the first that Google describes as designed specifically for inference. It was unveiled at Google Cloud Next on April 9, 2025. On November 6, 2025 Google announced that it would be generally available in the coming weeks; the Cloud TPU release notes record it entering preview on November 24, 2025 and reaching general availability on March 31, 2026 [9][16][19]. Each chip delivers 4,614 TFLOPS peak performance (FP8), a 10x improvement over TPU v5p per chip. Memory capacity jumps to 192 GB of HBM3E per chip with 7.37 TB/s bandwidth, six times the memory of Trillium.

Ironwood chips communicate via ICI at 9.6 Tb/s per chip. A full Ironwood superpod consists of 9,216 chips with access to 1.77 petabytes of aggregate HBM, delivering 42.5 FP8 exaflops per pod. Performance per watt is 2x that of Trillium, and Google states Ironwood is nearly 30x more power efficient than its first Cloud TPU from 2018. Each chip contains two TensorCores and four SparseCores [9][16][24]. Google describes Ironwood as "purpose-built to power thinking, inferential AI models at scale" and positions it for what it calls the "age of inference" [9].

### TPU 8t and TPU 8i

Google announced its eighth generation at Google Cloud Next on April 22, 2026 as two chips: TPU 8t for large-scale pre-training, with 9,600-chip superpods delivering 121 exaFLOPS of FP4 compute, and TPU 8i for post-training and inference, with 288 GB of HBM, 384 MB of on-chip SRAM and pods of up to 1,152 chips joined by a new Boardfly topology. Both run on Google's Arm-based Axion host CPUs, and Google rates them at up to twice Ironwood's performance per watt.[42][43] Google said both would be generally available later in 2026; in October 2026 its TPU product page still listed them as coming soon.[3][43]

## How are TPUs organized into pods, slices, and Multislice?

TPU hardware is organized into a hierarchy of groupings:

- **Chip:** A single TPU package containing TensorCores, SparseCores, HBM, and ICI interfaces.
- **Host:** A physical machine (TPU VM) connected to one or more TPU chips.
- **Slice:** A contiguous group of TPU chips within a pod connected via ICI. Users can provision slices of various sizes depending on their workload requirements.
- **Pod:** The largest contiguous grouping of TPU chips connected by ICI within a single physical installation; see [TPU Pod](https://aiwiki.ai/wiki/tpu_pod) for pod sizes and topologies by generation.

Cloud TPU Multislice is a scaling technology that allows a single training job to span multiple [TPU slice](https://aiwiki.ai/wiki/tpu_slice)s, even across different pods. Slices within a Multislice configuration communicate through data center networking (DCN), which has higher latency and lower throughput than ICI.[29] Multislice supports [data parallelism](https://aiwiki.ai/wiki/data_parallelism), Fully Sharded Data Parallelism (FSDP), [model parallelism](https://aiwiki.ai/wiki/model_parallelism), and [pipeline parallelism](https://aiwiki.ai/wiki/pipeline_parallelism). In November 2023 Google used it to run what it called the largest publicly disclosed LLM training job by chip count, on 50,944 TPU v5e chips [10].

## Software ecosystem and framework support

Cloud TPUs have supported three major machine learning frameworks, although TPU7x (Ironwood) runs only JAX and PyTorch:[30]

| Framework | Integration method | Notes |
|---|---|---|
| [JAX](https://aiwiki.ai/wiki/jax) | Native via XLA | Primary framework for TPU development; developed by [Google](https://aiwiki.ai/wiki/google); compiles Python and NumPy-like code to XLA |
| [TensorFlow](https://aiwiki.ai/wiki/tensorflow) | Native via XLA | Supported from TPU v2 through v6e; PJRT support added with TensorFlow 2.15.0; not supported on TPU7x |
| [PyTorch](https://aiwiki.ai/wiki/pytorch) | Via PyTorch/XLA; native TorchTPU in preview (2026) | Open-source library maintained by [Google](https://aiwiki.ai/wiki/google) and the PyTorch community; uses XLA as the compiler backend |

### JAX

JAX is a numerical computing library developed by Google that combines NumPy-like syntax with [automatic differentiation](https://aiwiki.ai/wiki/automatic_differentiation) and [XLA (Accelerated Linear Algebra)](https://aiwiki.ai/wiki/xla) compilation. JAX is the primary framework for TPU development at Google and is used for training large-scale models including Gemini. JAX's functional programming model maps naturally to TPU hardware, and its `pjit` and `shard_map` APIs provide fine-grained control over how computations and data are distributed across TPU chips [11][14].

### TensorFlow

TensorFlow was the original framework supported on Cloud TPUs. The TPU execution model in TensorFlow uses XLA compilation to translate TensorFlow graphs into optimized TPU machine code. Cloud TPU added support for the PJRT runtime interface with TensorFlow 2.15.0 in November 2023, and Google's documentation says TensorFlow is not supported on Ironwood (TPU7x).[19][30]

### PyTorch/XLA and TorchTPU

[PyTorch](https://aiwiki.ai/wiki/pytorch)/XLA is an open-source library that enables PyTorch models to run on TPUs by converting PyTorch operations into XLA HLO (High Level Operations) graphs. The torchax library from Google further bridges PyTorch and JAX by wrapping JAX arrays as PyTorch tensor subclasses, so the two can be mixed in one program.[65] More recently, [vLLM](https://aiwiki.ai/wiki/vllm) TPU (powered by tpu-inference) has unified JAX and PyTorch under a single lowering path for high-throughput LLM inference on TPUs.[64]

Reuters reported in December 2025 that Google was building a native PyTorch stack for TPUs, known internally as TorchTPU, with help from [Meta](https://aiwiki.ai/wiki/meta_ai), the steward of PyTorch.[54] In April 2026 Google said native PyTorch support for TPUs, including eager mode, was in preview, and in October 2026 its TPU product page described TorchTPU as an "eager-first" stack built with Meta.[3][42]

## How much do Cloud TPUs cost, and how do you access them?

Google Cloud offers several ways to provision and use TPUs:

- **On-demand:** Pay per chip-hour with no commitment. Offers maximum flexibility but highest per-unit cost.
- **Committed use discounts (CUDs):** 1-year or 3-year commitments that reduce per-chip-hour pricing significantly.
- **Spot and preemptible TPUs:** Reduced-price capacity that Google can reclaim at short notice, suited to fault-tolerant workloads. Google says Spot VM prices are variable and carry discounts of up to 91% off default prices for many machine types, GPUs and TPUs.[37]
- **Dynamic Workload Scheduler (DWS):** Flex-start, which entered preview in March 2025, provisions TPUs for up to 7 days through queued requests, and calendar mode, in preview from May 2025, reserves TPUs for 1 to 90 days.[19]
- **All Capacity mode:** For TPU7x on GKE, a reservation that gives full access to the reserved capacity and visibility into the hardware topology and health.[24]
- **TPU Research Cloud (TRC):** A program giving researchers free access to Cloud TPUs, in exchange for sharing their work through publications, open-source code or blog posts.[38]

### Pricing history

Google lists Cloud TPU prices per chip-hour. The following rates were listed on Google's pricing page; the v5e and v5p rates appear in an archived copy from May 2024, Trillium's from December 2024 and Ironwood's from March 2026, and all were unchanged when the page was checked in October 2026:[33][34][35][36]

| TPU type (region) | On-demand | 1-year commitment | 3-year commitment | DWS Flex-start (Oct 2026) | Spot (Oct 16, 2026) |
|---|---|---|---|---|---|
| TPU v4 pod (us-central2) | $3.22 | $2.0286 | $1.449 | N/A | $0.92 |
| [TPU v5e](https://aiwiki.ai/wiki/google_tpu_v5e) (us-central1) | $1.20 | $0.84 | $0.54 | $0.60 | $0.36 |
| [TPU v5p](https://aiwiki.ai/wiki/google_tpu_v5p) (us-east5) | $4.20 | $2.94 | $1.89 | $2.10 | $0.93 |
| TPU v6e Trillium (us-east1) | $2.70 | $1.89 | $1.22 | $1.35 | $0.58 |
| TPU7x Ironwood (us-central1) | $12.00 | $8.40 | $5.40 | $6.00 | $4.12 |

Spot rates are from Google's Spot VM price list (us-central1, except TPU v4 in us-central2) and change over time.[37] At the original Cloud TPU beta in February 2018, Google charged $6.50 per hour for a four-chip TPU v2 board.[20]

[TPU resource](https://aiwiki.ai/wiki/tpu_resource)s can be provisioned through the Google Cloud console, the `gcloud` CLI, Compute Engine, or Google Kubernetes Engine (GKE). Compute Engine's APIs for creating and managing TPU VMs became generally available on June 1, 2026, while the older Cloud TPU API now receives only bug fixes and security updates and does not support TPU7x.[19][41] For the latest TPU versions Google recommends Compute Engine or GKE,[41] and GKE provides job queueing with Kueue and Multislice job orchestration through the JobSet API.[39] In October 2026 TPU7x was offered in the us-central1 region, while Trillium was available in regions across North America, Europe, Asia and South America.[63]

## How do Cloud TPUs compare with GPUs?

TPUs and GPUs differ in their design philosophy and target workloads. The following table highlights the main differences:

| Aspect | Cloud TPU | GPU (e.g., NVIDIA H100/A100) |
|---|---|---|
| Design approach | Purpose-built ASIC for ML | General-purpose parallel processor |
| Precision formats | bfloat16, INT8, FP8 (v7), FP32 accum. | FP16, bfloat16, FP8, TF32, FP32, INT8 |
| Primary compute unit | [Systolic array](https://aiwiki.ai/wiki/systolic_array) (MXU) | [CUDA](https://aiwiki.ai/wiki/cuda) cores, [Tensor Core](https://aiwiki.ai/wiki/tensor_core)s |
| Memory type | HBM (integrated) | HBM (integrated) |
| Interconnect | ICI (custom, in-pod) | NVLink, NVSwitch, [InfiniBand](https://aiwiki.ai/wiki/infiniband) |
| Software ecosystem | [JAX](https://aiwiki.ai/wiki/jax), [TensorFlow](https://aiwiki.ai/wiki/tensorflow), [PyTorch](https://aiwiki.ai/wiki/pytorch)/XLA | [CUDA](https://aiwiki.ai/wiki/cuda), [cuDNN](https://aiwiki.ai/wiki/cudnn), all major frameworks |
| Vendor lock-in | Mainly [Google Cloud](https://aiwiki.ai/wiki/google_cloud); some third-party and customer data centers since 2025-2026 | Multi-cloud, on-premises |
| Strengths | Large-batch training, LLM inference, cost per FLOP | Flexibility, broad framework support, general-purpose compute |

Google's documentation recommends TPUs for models dominated by matrix computations, models that train for weeks or months, large models with large effective batch sizes, and models with ultra-large embeddings, and says they are not suited to linear algebra programs that branch frequently, workloads that need high-precision arithmetic, or neural networks with custom operations in the main training loop.[40] GPUs, by contrast, run NVIDIA's widely used CUDA software stack, are offered by every major cloud provider, and can be bought outright for on-premises use [12].

The choice between TPUs and GPUs often depends on the specific workload, scale, framework preference, and whether vendor portability is a priority.

## Who uses Cloud TPUs?

TPUs have powered many of Google's most notable AI systems and attracted major external customers:

- **AlphaGo, [AlphaZero](https://aiwiki.ai/wiki/alphazero) and [AlphaFold](https://aiwiki.ai/wiki/alphafold):** AlphaGo ran on TPUs in its 2016 matches against world champion Lee Sedol, AlphaZero used 5,000 first-generation and 64 second-generation TPUs for self-play and training in 2017, and AlphaFold 2 was trained on 128 TPU v3 cores; AlphaFold 2 (which solved the protein structure prediction problem) contributed to a Nobel Prize in Chemistry in 2024 [13][21][46][47].
- **Gemini:** Google's multimodal AI model family. Google's model card says [Gemini 3](https://aiwiki.ai/wiki/gemini_3) Pro was trained on TPUs using JAX and ML Pathways, without naming a TPU generation [14].
- **[Google Search](https://aiwiki.ai/wiki/google_search) and Translate:** TPU v1 was originally deployed to accelerate inference for Google Search ranking and the [neural machine translation](https://aiwiki.ai/wiki/neural_machine_translation) system behind Google Translate [1].
- **[Anthropic](https://aiwiki.ai/wiki/anthropic):** On October 23, 2025, Anthropic announced plans to expand its use of Google Cloud, including up to one million TPUs, in a deal worth tens of billions of dollars expected to bring well over a gigawatt of capacity online in 2026 [15]. Google Cloud CEO Thomas Kurian said the expansion reflected "the strong price-performance and efficiency" Anthropic's teams had seen with TPUs for several years [15]. On April 6, 2026, Anthropic signed a further agreement with Google and Broadcom for multiple gigawatts of next-generation TPU capacity starting in 2027, which Broadcom's filing put at about 3.5 gigawatts accessed through Broadcom and Anthropic later described as 5 gigawatts [17][60][62]. Later that month Google committed $10 billion in cash to Anthropic, with up to $30 billion more tied to performance targets [61]. The [Anthropic-Google TPU deal](https://aiwiki.ai/wiki/anthropic_google_tpu_deal) article covers these agreements in detail.
- **[Apple](https://aiwiki.ai/wiki/apple):** Apple trained its 2024 Apple Foundation Models on Cloud TPUs, the server model on 8,192 TPU v4 chips and the on-device model on 2,048 TPU v5p chips [44]. In January 2026 Apple and Google announced that the next generation of Apple Foundation Models would be based on Gemini models and Google's cloud technology, with Apple Intelligence continuing to run on Apple devices and Apple's Private Cloud Compute; DigiTimes reported that Apple would pay roughly $1 billion a year [18][45].
- **[Midjourney](https://aiwiki.ai/wiki/midjourney) and [Salesforce](https://aiwiki.ai/wiki/salesforce):** Google named the [AI image generation](https://aiwiki.ai/wiki/ai_image_generation) company and Salesforce, along with Anthropic, among the AI teams using Cloud TPUs intensively in 2024 [48].
- **Meta:** The Information reported in February 2026 that Meta had signed a multi-year, multibillion-dollar deal to rent Google TPUs; neither company commented [53].
- **Other named customers:** Two Sigma and Lyft used Cloud TPUs at the 2018 beta [20]; [Safe Superintelligence](https://aiwiki.ai/wiki/safe_superintelligence) uses Google Cloud TPUs for its research [50]; Lightricks and [Essential AI](https://aiwiki.ai/wiki/essential_ai) were early Ironwood users [16]; and Alphabet named Thinking Machines Lab, Hudson River Trading and Boston Dynamics as TPU customers in April 2026 [51]. [OpenAI](https://aiwiki.ai/wiki/openai) said in June 2025 that it was in early testing with some Google TPUs but had no plans to deploy them at scale [49].

## Are TPUs available outside Google Cloud?

Until 2025 TPUs ran only in Google's own data centers, either for Google's services or for Google Cloud customers. In September 2025 The Information reported that Google had begun placing TPUs in data centers run by smaller cloud providers, with [Fluidstack](https://aiwiki.ai/wiki/fluidstack) installing them at a new site in New York and Google approaching CoreWeave and Crusoe [55]. Google backstopped about $3.2 billion of Fluidstack's lease obligations at TeraWulf's Lake Mariner campus in New York, for warrants equal to about 14% of TeraWulf, and $1.4 billion at Cipher Mining's Barber Lake site in Colorado City, Texas, for about 5.4% of Cipher [56][57]. [SemiAnalysis](https://aiwiki.ai/wiki/semianalysis) reported in November 2025 that the first 400,000 Ironwood chips of Anthropic's deal, worth about $10 billion, would be sold by [Broadcom](https://aiwiki.ai/wiki/broadcom) to Anthropic as finished racks for its own facilities, with Fluidstack handling installation and TeraWulf and Cipher supplying the data centers [58].

Alphabet said on its April 2026 earnings call that it would begin delivering TPUs to a select group of customers in their own data centers, and in July 2026 that it had delivered TPU systems to customer data centers for the first time in the second quarter, with most of the revenue from these sales expected in 2027 [51][52]. In May 2026 Blackstone announced a joint venture with Google that will offer Google Cloud TPUs as a service from new data centers, with $5 billion of initial equity and the first 500 MW of capacity expected online in 2027 [59]. The Information also reported in February 2026 that Meta was in talks to buy TPUs for its own data centers [53].

## Limitations

Despite their strong performance for ML workloads, Cloud TPUs have several limitations:

- **Vendor lock-in:** TPUs are not offered by AWS or Azure, and for most customers they remain available only through Google Cloud; moving TPU workloads to other clouds or on-premises GPU hardware requires porting to GPU-compatible code [12]. Since 2025-2026 a small number of large customers and partners have hosted TPUs outside Google's data centers [52].
- **Framework constraints:** While JAX and PyTorch are supported, JAX remains the best-supported framework, and TensorFlow is not supported on TPU7x [30]. PyTorch has run through PyTorch/XLA, and Reuters reported in December 2025 that Google's native TorchTPU effort aims to remove a key barrier that has slowed TPU adoption [54].
- **Workload suitability:** TPUs excel at regular, dense matrix computation but may underperform on workloads with irregular memory access patterns, heavy branching, or operations not well-suited to systolic arrays [40].
- **Availability:** Demand for TPU capacity often exceeds supply, and quota allocation can be a bottleneck for new users; in July 2026 Alphabet described a supply-constrained environment and said it would use more third-party capacity as a bridge [52].
- **Debugging complexity:** Debugging performance issues on TPUs requires familiarity with XLA compilation, HLO graph analysis, and TPU-specific profiling tools, which have a steeper learning curve than CUDA-based GPU profiling.

## References

1. Jouppi, N.P., et al. "In-Datacenter Performance Analysis of a Tensor Processing Unit." Proceedings of the 44th Annual International Symposium on Computer Architecture (ISCA), 2017. https://arxiv.org/abs/1704.04760
2. Google Cloud. "BFloat16: The secret to high performance on Cloud TPUs." https://cloud.google.com/blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpus
3. Google Cloud. "Tensor Processing Units (TPUs)." https://cloud.google.com/tpu
4. Google Cloud. "TPU v3 documentation." https://docs.cloud.google.com/tpu/docs/v3
5. Jouppi, N.P., et al. "TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings." arXiv:2304.01433, 2023. https://arxiv.org/abs/2304.01433
6. Google Cloud. "TPU v5e documentation." https://docs.cloud.google.com/tpu/docs/v5e
7. Google Cloud Blog. "Introducing Cloud TPU v5p and AI Hypercomputer." https://cloud.google.com/blog/products/ai-machine-learning/introducing-cloud-tpu-v5p-and-ai-hypercomputer
8. Google Cloud Blog. "Introducing Trillium, sixth-generation TPUs." https://cloud.google.com/blog/products/compute/introducing-trillium-6th-gen-tpus
9. Google Blog. "Ironwood: The first Google TPU for the age of inference." https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/ironwood-tpu-age-of-inference/
10. Google Cloud Blog. "The world's largest distributed LLM training job on TPU v5e." https://cloud.google.com/blog/products/compute/the-worlds-largest-distributed-llm-training-job-on-tpu-v5e
11. JAX documentation. https://jax.readthedocs.io/
12. CNBC. "Gemini 3 - and the custom chips that power it - is a wake up call for AI investors." CNBC, 2025-11-25. https://www.cnbc.com/2025/11/25/gemini-3-and-the-custom-chips-that-train-it-is-a-wake-up-call-for-ai-investors.html
13. DeepMind. "AlphaFold: a solution to a 50-year-old grand challenge in biology." https://deepmind.google/research/breakthroughs/alphafold/
14. Google DeepMind. "Gemini 3 Pro Model Card." Google DeepMind, 2025-11-18. https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-3-Pro-Model-Card.pdf
15. Anthropic. "Expanding our use of Google Cloud TPUs and Services." https://www.anthropic.com/news/expanding-our-use-of-google-cloud-tpus-and-services
16. Google Cloud Blog. "Ironwood TPUs and new Axion-based VMs for your AI workloads." https://cloud.google.com/blog/products/compute/ironwood-tpus-and-new-axion-based-vms-for-your-ai-workloads
17. Anthropic. "Anthropic expands partnership with Google and Broadcom for multiple gigawatts of next-generation compute." Anthropic, 2026-04-06. https://www.anthropic.com/news/google-broadcom-partnership-compute
18. DigiTimes. "Apple taps Google Gemini to power Siri, pushing private cloud and Taiwan's AI supply chain." DigiTimes, 2026-01-16. https://www.digitimes.com/news/a20260116PD236/apple-google-gemini-siri-supply-chain.html
19. Google Cloud Documentation. "Cloud TPU release notes." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/release-notes
20. Google Cloud Blog. "Cloud TPU machine learning accelerators now available in beta." Google Cloud, 2018-02-13. https://cloud.google.com/blog/products/gcp/cloud-tpu-machine-learning-accelerators-now-available-in-beta/
21. Google Cloud Blog. "Google supercharges machine learning tasks with TPU custom chip." Google, 2016-05-19. https://cloud.google.com/blog/products/ai-machine-learning/google-supercharges-machine-learning-tasks-with-custom-chip
22. Google Cloud Blog. "An in-depth look at Google's first Tensor Processing Unit (TPU)." Google Cloud, 2017-05-13. https://cloud.google.com/blog/products/ai-machine-learning/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu
23. Google Cloud Documentation. "TPU architecture." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm
24. Google Cloud Documentation. "TPU7x (Ironwood)." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/tpu7x
25. Google Cloud Documentation. "TPU v6e." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/v6e
26. Google Cloud Documentation. "TPU v5p." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/v5p
27. Google Cloud Documentation. "TPU v4." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/v4
28. JAX Scaling Book. "How to Think About TPUs." Google DeepMind, 2026-10-16. https://jax-ml.github.io/scaling-book/tpus/
29. Google Cloud Documentation. "Cloud TPU Multislice overview." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/multislice-introduction
30. Google Cloud Documentation. "TPU software versions." Google Cloud, 2026-10-16. https://docs.cloud.google.com/tpu/docs/runtimes
31. Wikipedia. "Tensor Processing Unit." Wikipedia, 2026-10-16. https://en.wikipedia.org/wiki/Tensor_Processing_Unit
32. Techzine. "Google's machine learning services get a boost from new TPU v4." Techzine, 2021-05-25. https://www.techzine.eu/news/cloud/60481/googles-machine-learning-services-get-a-boost-from-new-tpu-v4/
33. Google Cloud. "Cloud TPU pricing." Google Cloud, 2026-10-16. https://cloud.google.com/tpu/pricing
34. Internet Archive. "Cloud TPU pricing (archived copy)." Internet Archive, 2024-05-15. https://web.archive.org/web/20240515164412/https://cloud.google.com/tpu/pricing
35. Internet Archive. "Cloud TPU pricing (archived copy)." Internet Archive, 2024-12-14. https://web.archive.org/web/20241214072327/https://cloud.google.com/tpu/pricing
36. Internet Archive. "Cloud TPU pricing (archived copy)." Internet Archive, 2026-03-09. https://web.archive.org/web/20260309161836/https://cloud.google.com/tpu/pricing
37. Google Cloud. "Spot VMs pricing." Google Cloud, 2026-10-16. https://cloud.google.com/spot-vms/pricing
38. Google Research. "TPU Research Cloud." Google, 2026-10-16. https://sites.research.google/trc/about/
39. Google Cloud Documentation. "About TPUs in GKE." Google Cloud, 2026-10-16. https://docs.cloud.google.com/kubernetes-engine/docs/concepts/tpus
40. Google Cloud Documentation. "Introduction to Cloud TPU." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/intro-to-tpu
41. Google Cloud Documentation. "Manage TPU Spot VMs." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/spot
42. Google Cloud Blog. "TPU 8t and TPU 8i technical deep dive." Google Cloud, 2026-04-23. https://cloud.google.com/blog/products/compute/tpu-8t-and-tpu-8i-technical-deep-dive
43. Google Blog. "Our eighth generation TPUs: two chips for the agentic era." Google, 2026-04-22. https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/eighth-generation-tpu-agentic-era/
44. Apple. "Apple Intelligence Foundation Language Models." arXiv, 2024-07-29. https://arxiv.org/abs/2407.21075
45. Google. "Joint statement from Google and Apple." Google, 2026-01-12. https://blog.google/company-news/inside-google/company-announcements/joint-statement-google-apple/
46. Silver, D., et al. "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm." arXiv, 2017-12-05. https://arxiv.org/abs/1712.01815
47. Jumper, J., et al. "Highly accurate protein structure prediction with AlphaFold." Nature, 2021-07-15. https://www.nature.com/articles/s41586-021-03819-2
48. Google Cloud Blog. "TPU transformation: A look back at 10 years of our AI-specialized chips." Google Cloud, 2024-08-01. https://cloud.google.com/transform/ai-specialized-chips-tpu-history-gen-ai
49. Reuters. "OpenAI sticks with Nvidia, holds off on Google's AI chips." Business Today, 2025-07-01. https://www.businesstoday.in/technology/news/story/openai-sticks-with-nvidia-holds-off-on-googles-ai-chips-482534-2025-07-01
50. TechCrunch. "Ilya Sutskever taps Google Cloud to power his AI startup's research." TechCrunch, 2025-04-09. https://techcrunch.com/2025/04/09/ilya-sutskever-taps-google-cloud-to-power-his-ai-startups-research
51. Alphabet. "Q1 2026 Earnings Call Transcript." Alphabet Investor Relations, 2026-04-29. https://s206.q4cdn.com/479360582/files/doc_events/2026/Apr/29/2026_Q1_Earnings_Transcript.pdf
52. Alphabet. "Q2 2026 Earnings Call Transcript." Alphabet Investor Relations, 2026-07-22. https://s206.q4cdn.com/479360582/files/doc_events/2026/Jul/22/2026_Q2_Earnings_Transcript.pdf
53. Reuters. "Meta signs multi-billion-dollar deal to rent Google AI chips, The Information reports." The Standard, 2026-02-27. https://www.thestandard.com.hk/tech-and-startup/article/325426/Meta-signs-multi-billion-dollar-deal-to-rent-Google-AI-chips-The-Information-reports/
54. Reuters. "Google works to erode Nvidia's software advantage with Meta's help." TimesLIVE, 2025-12-18. https://www.timeslive.co.za/news/sci-tech/2025-12-18-google-works-to-erode-nvidias-software-advantage-with-metas-help/
55. Business Standard. "Google pushes its AI chips into rival data centres, challenging Nvidia." Business Standard, 2025-09-08. https://www.business-standard.com/world-news/google-tpus-challenge-nvidia-ai-chips-cloud-data-centres-125090800159_1.html
56. TeraWulf. "TeraWulf Announces Fluidstack Expansion with 160 MW CB-5 Lease at Lake Mariner." U.S. Securities and Exchange Commission, 2025-08-18. https://www.sec.gov/Archives/edgar/data/1083301/000110465925079463/tm2523651d3_ex99-1.htm
57. Cipher Mining. "Cipher Mining Signs 168 MW, 10-Year AI Hosting Agreement with Fluidstack." U.S. Securities and Exchange Commission, 2025-09-25. https://www.sec.gov/Archives/edgar/data/1819989/000095010325012168/dp234624_ex9901.htm
58. SemiAnalysis. "Google TPUv7: The 900lb Gorilla In the Room." SemiAnalysis, 2025-11-28. https://newsletter.semianalysis.com/p/tpuv7-google-takes-a-swing-at-the
59. Blackstone. "Blackstone Announces Joint Venture with Google to Create New TPU Cloud." Business Wire, 2026-05-18. https://www.businesswire.com/news/home/20260518367552/en/Blackstone-Announces-Joint-Venture-with-Google-to-Create-New-TPU-Cloud
60. Broadcom Inc. "Current Report on Form 8-K (Item 8.01)." U.S. Securities and Exchange Commission, 2026-04-06. https://www.sec.gov/Archives/edgar/data/1730168/000119312526144028/d87999d8k.htm
61. Reuters. "Google to invest up to US$40B in AI rival Anthropic." BNN Bloomberg, 2026-04-24. https://www.bnnbloomberg.ca/business/company-news/2026/04/24/google-to-invest-up-to-us40b-in-ai-rival-anthropic/
62. Anthropic. "Higher usage limits and a SpaceX compute deal." Anthropic, 2026-05-06. https://www.anthropic.com/news/higher-limits-spacex
63. Google Cloud Documentation. "TPU regions and zones." Google Cloud, 2026-10-13. https://docs.cloud.google.com/tpu/docs/regions-zones
64. Google Cloud Blog. "Inside the Ironwood TPU codesigned AI stack." Google Cloud, 2025-11-06. https://cloud.google.com/blog/products/compute/inside-the-ironwood-tpu-codesigned-ai-stack
65. Google. "torchax." GitHub, 2026-10-16. https://github.com/google/torchax