TPU Pod

See also: Tensor Processing Unit (TPU), Cloud TPU, Data parallelism, Model parallelism, Distributed training

Overview

A TPU Pod is a single Google supercomputer built from many Tensor Processing Unit (TPU) chips wired directly to each other by a high-speed Inter-Chip Interconnect (ICI) fabric arranged as a 2D or 3D torus, so that software can treat the whole cluster as a single unit for machine learning training and inference.^[7][17] Pod size has grown across Google's performance-class TPUs: a TPU v4 Pod holds 4,096 chips, a TPU v5p Pod holds 8,960 chips, and the seventh-generation Ironwood (TPU v7) superpod holds 9,216 chips delivering 42.5 exaFLOPS of FP8 compute, the largest TPU Pod available as of October 2026; Google announced 9,600-chip superpods for its eighth-generation TPU 8t in April 2026.^{[2][9][12][23]} TPU Pods are the machines on which Google trains its largest models, including PaLM and the Gemini family.^[4][35]

Google describes the top Ironwood configuration in concrete terms: "When scaled to 9,216 chips per pod for a total of 42.5 Exaflops, Ironwood supports more than 24x the compute power of the world's largest supercomputer," which Google identified as El Capitan.^[12] That comparison sets Ironwood's FP8 peak against El Capitan's 64-bit LINPACK result.^[18] Rather than treating each chip as a separate device, a TPU Pod lets the XLA compiler view the entire cluster as one unified computational resource, enabling efficient distributed training of large language models and other compute-intensive AI systems.

Google introduced the TPU Pod concept with TPU v2 in 2017, when the company first connected multiple TPU chips via custom ICI links arranged in a 2D torus topology.^[14] Since then, each performance-class TPU generation has expanded Pod scale and interconnect sophistication. Early Pods contained 256 chips; Ironwood (TPU v7), the newest generally available generation, scales to 9,216 chips per Pod and delivers 42.5 exaFLOPS of aggregate FP8 compute.^[12] TPU Pods have been used to train many of the most influential AI models of the past decade, including PaLM, Gemini, and BERT.^[4][31][35]

A TPU Pod differs from a conventional GPU cluster in a fundamental way: the chips communicate through a dedicated, low-latency torus network rather than through general-purpose data center switches. This design enables collective operations (such as all-reduce) to run with minimal overhead, which is particularly important when thousands of chips must synchronize gradient updates during training.

ELI5 (Explain like I'm 5)

Imagine you have a giant jigsaw puzzle with millions of pieces. If you try to solve it by yourself, it could take weeks. But what if you got a whole classroom of friends to help? Each friend works on a section of the puzzle, and whenever they need a piece from someone else's section, they can pass it over quickly because they are all sitting at the same big table.

A TPU Pod works the same way. Each TPU chip is like one of those friends, and the special wires connecting them (called the "Inter-Chip Interconnect") are like the table that lets them pass puzzle pieces to each other really fast. Because the chips are all connected directly to their neighbors, they can share information almost instantly, so the whole group finishes the puzzle much faster than any single chip could on its own.

How did the TPU Pod evolve across generations?

TPU v1: single-chip inference (2015)

The first TPU was an inference-only chip with no inter-chip network. Each TPU v1 sat on its own PCIe card inside a server, operating independently.^[1] There was no concept of a "Pod" at this stage; the v1 was designed to accelerate neural network inference for production services such as Google Search, Google Translate, and the AlphaGo system.^[1]

TPU v2: the first Pods (2017)

Announced at Google I/O in May 2017, TPU v2 was the first generation to support both training and inference.^[14] It was also the first to introduce the Inter-Chip Interconnect (ICI), a custom high-speed bidirectional link connecting each chip directly to four neighbors in a 2D torus topology. Groups of four chips were packaged into modules delivering 180 TFLOPS. Sixty-four of these modules formed a 256-chip Pod with a peak throughput of approximately 11.5 petaFLOPS.^[14][20]

The 2D torus meant that each chip could communicate with its north, south, east, and west neighbors, and wraparound links connected chips on opposite edges of the grid. This was the architectural foundation on which all subsequent Pods were built.

TPU v3: scaling to 1,024 chips (2018)

TPU v3 retained the 2D torus ICI topology but increased per-link bandwidth and raised per-chip performance to 123 TFLOPS (bf16), 2.7 times v2's 45 TFLOPS. Pods scaled from 256 to 1,024 chips, and the aggregate Pod throughput exceeded 100 petaFLOPS (Google's documentation now lists 126 petaFLOPS in bf16).^[14][22] The higher power density of v3 chips required liquid cooling for the first time in Google's TPU program. In the MLPerf v0.6 training round of July 2019, Google's largest ResNet-50, SSD and Transformer submissions used full 1,024-chip TPU v3 Pods, and Google said it set three performance records.^[25][26]

TPU v4: 3D torus and optical switches (2021)

TPU v4 was a major architectural leap. The interconnect moved from a 2D torus to a 3D torus, where each chip connects to six neighbors along three axes instead of four neighbors in two axes. This lowers the network diameter: in a torus a message only has to travel halfway around each ring, so the worst-case path in a square 2D torus of N chips is about the square root of N hops, against about 1.5 times the cube root of N in a cubic 3D torus.^[24]

The most significant innovation was the introduction of Optical Circuit Switches (OCS); the TPU v4 paper describes v4 as the first supercomputer built with a reconfigurable interconnect that enhances performance.^[2] Pods scaled to 4,096 chips and delivered 1.1 exaFLOPS (bf16).^[2] The TPU v4 system was described in a paper presented at ISCA 2023.^[2]

TPU v5e and v5p: cost and performance tiers (2023)

Google split the fifth generation into two products. TPU v5e was a cost-efficient chip using a 2D torus with Pods of up to 256 chips, aimed at inference and moderate-scale training. TPU v5p was the performance variant, scaling to 8,960 chips in a 3D torus and delivering about 4.1 exaFLOPS of bf16 compute across a full Pod (8,960 chips at 459 TFLOPS each).^[9][11] Google states that each v5p Pod "composes together 8,960 chips over our highest-bandwidth inter-chip interconnect (ICI) at 4,800 Gbps/chip in a 3D torus topology."^[11] In MLPerf Training, Google entered a 4,096-chip TPU v5e system in round v3.1 (November 2023) and a 6,144-chip TPU v5p system in v4.0 (June 2024).^[27][28]

TPU v6e / Trillium (2024)

Trillium doubled ICI bandwidth over v5e and achieved 918 TFLOPS per chip (bf16). Pods scale to 256 chips in a 2D torus.^[21] Google reported that 100,000 Trillium chips can be connected within a single Jupiter data center fabric with 13 petabits per second of bisection bandwidth.^[20] Google entered Trillium systems of up to 3,072 chips, as a preview, in MLPerf Training v4.1 in November 2024.^[29]

TPU v7 / Ironwood (2025)

Ironwood represents the largest TPU Pod Google offers as of October 2026.^[12] A single rack of hosts contains 64 chips arranged as a 4x4x4 "cube" connected in a 3D torus. Multiple cubes are linked through OCS connections: Google offers a 256-chip pod built from four cubes and a 9,216-chip superpod built from 144 cubes, with 42.5 exaFLOPS of FP8 compute.^[12][13][16] Each Ironwood chip carries 192 GB of HBM3E and an ICI bandwidth of 9.6 Tb/s (1.2 TB/s) per chip, and a full superpod exposes about 1.77 PB of directly accessible HBM.^[12][13][16] The superpod is slightly larger than a power of two so that it holds spare racks: according to ServeTheHome's report of Google's Hot Chips 2025 talk, 8,192 chips would have left no spares.^[19] Beyond one superpod, Google links superpods over its Jupiter data-center network into clusters of hundreds of thousands of chips.^[17]

TPU 8t and TPU 8i (announced 2026)

Google's eighth generation, announced in April 2026 and due for general availability later that year, splits Pods by workload. TPU 8t keeps the 3D torus but grows the superpod to 9,600 chips, with about 2 PB of shared HBM, 121 exaFLOPS of FP4 compute and double Ironwood's interchip bandwidth; Google's new Virgo network can link more than 134,000 TPU 8t chips in one non-blocking fabric, and Google says JAX and Pathways can scale a training cluster to more than a million chips.^[23][24] TPU 8i uses a new hierarchical topology called Boardfly instead of a torus: four fully connected chips form a board, eight boards form a group, and 36 groups linked by optical circuit switches make a pod of up to 1,152 chips (1,024 active), with any chip at most seven hops from any other.^[24]

What are the Pod specifications by generation?

The following table summarizes key Pod-level specifications across TPU generations:

Sources: Google's documentation for v3 through TPU7x and its April 2026 announcements for the eighth generation; v5p's Pod peak is computed from its per-chip figure.^{[9][13][20][21][22][23][24]}

How does the interconnect architecture work?

Inter-Chip Interconnect (ICI)

The ICI is a custom high-speed link that directly connects neighboring TPU chips.^[7] Unlike GPU clusters that route inter-accelerator traffic through PCIe switches, NVLink bridges, or InfiniBand fabrics, ICI provides a direct, low-latency chip-to-chip path with terabit-per-second bandwidth.^[10][15] No host CPU is involved in ICI communication: Google describes ICI as supporting remote direct memory access that lets chips exchange data while bypassing the host CPU.^[16]

Key characteristics of ICI:

• Nearest-neighbor connectivity. Each chip connects only to its immediate neighbors in the torus. In a 2D torus, each chip has 4 ICI links; in a 3D torus, each chip has 6.^[8][21]
• Multi-hop routing. Communication between non-adjacent chips is routed through intermediate chips. The torus wraparound links cut the maximum hop count in half compared to a simple mesh.^[15]
• Bandwidth hierarchy. On TPU v5p, for example, each ICI link carries about 90 GB/s in each direction, and Google lists 1,200 GB/s of bidirectional ICI bandwidth per chip. This is well below the chip's HBM bandwidth (2,765 GB/s) but far above its data center network bandwidth (about 6.25 GB/s per chip).^[9][15]

2D torus topology

TPU v2, v3, v5e, and v6e use a 2D torus. In this layout, chips are arranged in a rectangular grid with wraparound links connecting each edge to the opposite edge. Each chip connects to four neighbors: north, south, east, and west.

The 2D torus Pods of TPU v5e and v6e are 16x16 (256 chips), while TPU v3 Pods were 32x32 (1,024 chips).^[15][21] This topology is simpler to program and sufficient for workloads that can be partitioned along two dimensions, such as data parallelism across one axis and model parallelism across the other.

3D torus topology

TPU v4, v5p, and v7 (Ironwood) use a 3D torus, adding a third axis with wraparound links. Each chip connects to six neighbors. The three-dimensional layout provides several advantages over a 2D torus:

• Lower network diameter. Because a message only has to travel halfway around each ring, the worst case in a k x k x k torus is about 1.5k hops (1.5 times the cube root of N), against about k hops for a square 2D torus of N = k x k chips. At 4,096 chips, that is about 24 hops for a 16x16x16 torus, compared with about 64 for a 64x64 one. Google used the same arithmetic in 2026 to put an 8x8x16 torus of 1,024 chips at 16 hops.^[24]
• Higher bisection bandwidth. The 3D torus has more links crossing any bisecting plane, providing greater bandwidth for collective operations like all-reduce.
• Three axes for parallelism. The three physical dimensions can be mapped to data parallelism, model (tensor) parallelism, and pipeline parallelism simultaneously.

The basic building block of a 3D torus Pod is a cube of 4x4x4 = 64 chips. Cubes are connected to each other via wraparound links, which in TPU v4, v5p and Ironwood pass through Optical Circuit Switches.^[2][7]

Twisted torus

TPU v4 introduced a twisted torus variant, where the wraparound links are shifted so that a chip at position (x, y, z) connects not to position (x, y, 0) on the opposite edge but to a position offset by a fixed amount.^[2] The twisted torus increases bisection bandwidth by approximately 70% for certain slice shapes (for example, a 4x4x8 twisted torus vs. a standard 4x4x8 torus).^[2][8] Because the twist is implemented through OCS routing tables rather than physical rewiring, users can choose between a standard torus and a twisted torus for any given workload; Google's documentation lists twisted topologies for TPU v4 and v5p.^[2][9]

Optical Circuit Switches (OCS)

OCS technology is one of the most distinctive features of TPU v4, v5p and Ironwood Pods.^[2][7] An OCS uses arrays of tiny mirrors built with Micro-Electro-Mechanical Systems (MEMS) technology to steer optical signals between fiber-optic cables.^[2][6] Switching happens in milliseconds, and the switch steers light directly rather than converting the signal back to electrical form.^[2][6]

The OCS layer sits between cubes. Within a cube, chips are connected via direct electrical (copper) ICI links in a 3D mesh. The OCS provides the wraparound links that turn this mesh into a torus, and it can dynamically reconfigure which cubes are connected to each other.^[7]

Benefits of OCS include:

Data center network (DCN)

For workloads that span multiple Pods or multiple slices, TPU hosts communicate through Google's data center network.^[10] DCN bandwidth per chip is much lower than ICI bandwidth (roughly 6.25 GB/s per chip on v5p, compared with 1,200 GB/s of bidirectional ICI bandwidth), so the XLA compiler and runtime schedule DCN communication carefully to overlap it with computation.^[9][10][15]

What are slices and multislice?

Slices

A slice is a contiguous set of TPU chips within a single Pod, all connected via ICI. Slices are the unit of allocation in Cloud TPU: when a user requests TPU resources, they receive a slice of a specific topology (for example, a v4-128 slice is a 4x4x4 cube of 64 chips assigned as 128 TensorCores, since each v4 chip has two TensorCores).^[7][8]

Slice sizes vary by TPU generation. TPU v4 slice names count TensorCores, so they run from v4-8 (4 chips in a 2x2x1 shape) upward, with v4-4096 being a 2,048-chip 8x16x16 slice; slices above 64 chips come in increments of 64 chips.^[8] For TPU v5p, configurations range from 4 chips (2x2x1) up to 6,144 chips (16x16x24), which is the largest schedulable job size; the full v5p Pod has 140 cubes, against the 96 cubes of the largest slice.^[9]

Multislice

Cloud TPU Multislice is a scaling technology that allows a single training job to span multiple slices. Chips within each slice communicate via ICI as usual, while chips in different slices exchange data through DCN by routing traffic through host CPUs.^[10]

The XLA compiler automatically generates the inter-slice DCN communication code. Developers do not need to write explicit networking logic; the compiler inserts the necessary collective operations and overlaps them with computation to hide latency.^[10]

Multislice scaling has been demonstrated at very large scale. In November 2023 Google ran what it believed was the largest publicly disclosed LLM training job by chip count, training models of 16 billion to 128 billion parameters on a cluster of 50,944 TPU v5e chips (199 v5e Pods).^[10] Among Multislice's benefits, Google lists: "Train massive models with near-linear scaling performance from single slice to multiple slices with up to tens of thousands of chips."^[30] Apple trained its 2024 server foundation model on 8,192 TPU v4 chips provisioned as eight 1,024-chip slices connected over the data center network.^[37]

How are TPU Pods programmed?

XLA and GSPMD

The primary compiler for TPU Pods is XLA (Accelerated Linear Algebra), a domain-specific compiler that translates high-level ML operations into optimized TPU machine code. XLA performs whole-program analysis, fusing operations, tiling computations to fit in on-chip memory, and scheduling data transfers to keep the hardware busy.

For distributed workloads on TPU Pods, XLA uses GSPMD (General-purpose Single Program Multiple Data), a partitioning pass that automatically shards a computation across all chips in the Pod. Developers annotate tensors with sharding specifications (for example, "shard this tensor's batch dimension across the first axis of the Pod"), and GSPMD transforms the single-device program into a distributed one, inserting the correct collective communication operations.^[5]

This approach means developers can write code as if it will run on a single large device. The compiler handles:

• Partitioning the model and data across chips
• Inserting all-reduce, all-gather, and reduce-scatter collectives
• Overlapping communication with computation
• Mapping logical tensor dimensions to physical Pod topology axes

Framework support

TPU Pods are supported by three major ML frameworks, although Ironwood supports only JAX and PyTorch:^[13]

JAX is the most commonly used framework for TPU Pod workloads at Google. Its functional programming model aligns naturally with XLA's compilation requirements, and JAX transformations like jit, vmap, pmap, and shmap map directly to Pod topologies.^[5]

Parallelism strategies

TPU Pods support all standard distributed training strategies:

• Data parallelism. The model is replicated across chips, and each chip processes a different mini-batch. Gradients are synchronized via all-reduce over ICI. This is the simplest and most common strategy.
• Model parallelism (tensor parallelism). Individual layers are split across chips, with each chip holding a slice of the weight tensors. Activations are communicated between chips during the forward and backward passes.
• Pipeline parallelism. Different layers of the model are assigned to different groups of chips. Micro-batches flow through the pipeline, with each group processing a different micro-batch at any given time.
• Expert parallelism. Used in Mixture of Experts (MoE) architectures, where different expert sub-networks reside on different chips and a gating network routes tokens to the appropriate expert.

In practice, large-scale training on TPU Pods combines multiple strategies. For example, PaLM 540B used a combination of data parallelism and model parallelism across 6,144 TPU v4 chips spanning two Pods.^[4]

How do TPU Pods handle hardware failures?

Operating thousands of chips continuously for days or weeks of training requires robust fault-handling mechanisms. At the scale of a TPU v4 Pod, hardware failures are not rare events but routine occurrences.^[3]

Failure rates

A study by Zu et al., presented at NSDI 2024, reported the following average daily failure rates for the components of Google's TPU v4 supercomputers:^[3]

A 4,096-chip TPU v4 Pod has 1,024 four-chip machines, so the machine failure rate works out to just under one machine failure per day; the paper notes that because each supercomputer runs many jobs at once, the number of jobs affected by hardware outages is still significant.^[3][4] Without automated recovery, such failures would cause frequent training interruptions.

Automated recovery

Google's TPU infrastructure uses several techniques to maintain high availability:

• OCS reconfiguration. When the fabric manager detects a failed chip or link, it reconfigures the optical switches to route around the fault, connecting healthy cubes into a new slice that excludes the defective hardware. This happens without human intervention.^[3]
• ICI resiliency. For TPU v4, v5p, and Ironwood, ICI connections can be routed around OCS and optical ICI faults, maintaining slice availability with only temporary performance degradation.^[7]
• Spare capacity. Pods hold more cubes than the largest schedulable slice (a v5p Pod has 140 cubes, while its largest slice uses 96), and on Ironwood the OCS fabric manager swaps in a designated spare when a cube or link fails.^[9][16]
• Checkpointing. Training frameworks save model state to persistent storage at regular intervals. When a failure occurs, training restarts from the most recent checkpoint rather than from scratch.
• In-memory redundancy. For training Gemini, Google used redundant in-memory copies of model state distributed across replicas. On hardware failure, intact replicas provided the model state for recovery, avoiding the latency of reading checkpoints from storage.^[35]

The Zu et al. study reported that TPU v4 supercomputers achieve 99.98% system availability through these automated mechanisms, with hardware outages affecting approximately 1% of training jobs.^[3]

What models are trained on TPU Pods?

The following table lists major models and systems trained on TPU Pods:

Sources: the model papers and reports listed in the references.^{[4][31][32][33][34][35][36][37]}

PaLM is a particularly instructive example of TPU Pod usage. The 540B-parameter model was trained across two TPU v4 Pods, each with 3,072 chips (for a total of 6,144 chips), using the Pathways system to coordinate computation across Pods. The training achieved 57.8% hardware FLOPS utilization, the highest figure reported for LLM training at that scale at the time of publication.^[4]

How do TPU Pods differ from GPU clusters?

The primary alternative to TPU Pods for large-scale ML training is NVIDIA GPU clusters connected via NVLink and InfiniBand. The two approaches differ in several respects:

Sources for the scale and availability rows: Google, NVIDIA and Alphabet.^{[12][23][41][43]}

TPU Pods have a structural advantage in that ICI provides a uniform, high-bandwidth fabric across the entire Pod without the bandwidth bottleneck that occurs between nodes in GPU clusters.^[15] In a GPU cluster, NVLink provides very high bandwidth within a single multi-GPU node (e.g., 900 GB/s per GPU in DGX H100), but communication between nodes drops to InfiniBand speeds (typically 400 Gb/s per port). In a TPU Pod, every chip-to-chip link uses ICI regardless of physical distance within the Pod.

GPU clusters, on the other hand, benefit from a much larger and more mature software ecosystem, broader availability across cloud providers, and the ability to be purchased as on-premises hardware.

How do you access TPU Pods on Google Cloud?

Pricing

Cloud TPU Pods are billed per chip-hour, and prices vary by generation, region, and commitment level. In October 2026 Google's pricing page listed on-demand rates of $1.20 per chip-hour for TPU v5e, $4.20 for TPU v5p, $2.70 for Trillium (us-east1) and $12.00 for Ironwood (us-central1); one-year and three-year commitments cut those rates by about 30% and 55% respectively.^[38] The Cloud TPU article lists the full, dated price history.

Google also offers Spot capacity at variable discounts for fault-tolerant workloads, Dynamic Workload Scheduler options (Flex-start and calendar-mode reservations), and queued resources for users who can wait for availability.^[20][42]

Regional availability

TPU Pods are available in select Google Cloud regions, and not every generation is offered everywhere. In October 2026 Google's zone list showed Ironwood (TPU7x) only in us-central1, v5p in us-central1, us-east5 and europe-west4, and Trillium in zones across North America, Europe, Asia and South America.^[39] Some capacity requires manual approval: Google grants no default quota for TPU v4 in us-central2-b, so all requests there are reviewed by Google.^[8]

TPU Research Cloud (TRC)

Google's TPU Research Cloud program provides free Cloud TPU access to researchers. In exchange, participants are expected to share their work through peer-reviewed publications, open-source code, blog posts, or other means.^[40]

See also

• Tensor Processing Unit (TPU)
• Cloud TPU
• TPU Ironwood
• TPU Board

References

1. ^1 ^2Jouppi, N. P., et al. "In-Datacenter Performance Analysis of a Tensor Processing Unit." *Proceedings of the 44th Annual International Symposium on Computer Architecture (ISCA)*, 2017. arXiv:1704.04760
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Jouppi, N. P., et al. "TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings." *Proceedings of the 50th Annual International Symposium on Computer Architecture (ISCA)*, 2023. arXiv:2304.01433
3. ^1 ^2 ^3 ^4 ^5Zu, Y., et al. "Resiliency at Scale: Managing Google's TPUv4 Machine Learning Supercomputer." *Proceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI)*, 2024. usenix.org/conference/nsdi24/presentation/zu
4. ^1 ^2 ^3 ^4 ^5 ^6Chowdhery, A., et al. "PaLM: Scaling Language Modeling with Pathways." *arXiv preprint*, 2022. arXiv:2204.02311
5. ^1 ^2Yoo, J., et al. "Scalable Training of Language Models using JAX pjit and TPUv4." *arXiv preprint*, 2022. arXiv:2204.06514
6. ^1 ^2Patel, D., et al. "Lightwave Fabrics: At-Scale Optical Circuit Switching for Datacenter and Machine Learning Systems." *Proceedings of the ACM SIGCOMM Conference*, 2023. dl.acm.org/doi/10.1145/3603269.3604836
7. ^1 ^2 ^3 ^4 ^5 ^6 ^7Google Cloud. "TPU system architecture." *Cloud TPU Documentation*. docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm
8. ^1 ^2 ^3 ^4 ^5Google Cloud. "TPU v4 documentation." *Cloud TPU Documentation*. docs.cloud.google.com/tpu/docs/v4
9. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Google Cloud. "TPU v5p documentation." *Cloud TPU Documentation*. docs.cloud.google.com/tpu/docs/v5p
10. ^1 ^2 ^3 ^4 ^5 ^6Google Cloud. "Cloud TPU Multislice overview" and "The world's largest distributed LLM training job on TPU v5e." *Cloud TPU Documentation and Google Cloud Blog*. docs.cloud.google.com/tpu/docs/multislice-introduction, cloud.google.com/blog/products/compute/the-worlds-largest-distributed-llm-training-job-on-tpu-v5e
11. ^1 ^2Google Cloud. "Introducing Cloud TPU v5p and AI Hypercomputer." *Google Cloud Blog*, 2023. cloud.google.com/blog/products/ai-machine-learning/introducing-cloud-tpu-v5p-and-ai-hypercomputer
12. ^1 ^2 ^3 ^4 ^5 ^6 ^7Google. "Ironwood: The first Google TPU for the age of inference." *The Keyword*, 2025. blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/ironwood-tpu-age-of-inference
13. ^1 ^2 ^3 ^4Google Cloud. "TPU7x (Ironwood) documentation." *Cloud TPU Documentation*. docs.cloud.google.com/tpu/docs/tpu7x
14. ^1 ^2 ^3 ^4Wikipedia. "Tensor Processing Unit." en.wikipedia.org/wiki/Tensor_Processing_Unit
15. ^1 ^2 ^3 ^4 ^5 ^6Scaling Book (JAX). "How to Think About TPUs." jax-ml.github.io/scaling-book/tpus
16. ^1 ^2 ^3 ^4Google Cloud Blog. "Inside the Ironwood TPU codesigned AI stack." Google Cloud, 2025-11-06. cloud.google.com/...onwood-tpu-codesigned-ai-stack
17. ^1 ^2Google Cloud Blog. "Ironwood TPUs and new Axion-based VMs for your AI workloads." Google Cloud, 2025-11-07. cloud.google.com/...ased-vms-for-your-ai-workloads
18. ^The Register. "Google's 7th-gen Ironwood TPUs promise 42 AI exaFLOPS pods." The Register, 2025-04-10. theregister.com/...gles_7thgen_ironwood_tpus_debut
19. ^ServeTheHome. "Google Ironwood TPU Swings for Reasoning Model Leadership at Hot Chips 2025." ServeTheHome, 2025-08-27. servethehome.com/...l-leadership-at-hot-chips-2025
20. ^1 ^2 ^3 ^4Google Cloud Documentation. "Cloud TPU release notes." Google Cloud, 2026-10-13. docs.cloud.google.com/...release-notes
21. ^1 ^2 ^3 ^4Google Cloud Documentation. "TPU v6e." Google Cloud, 2026-10-13. docs.cloud.google.com/...v6e
22. ^1 ^2Google Cloud Documentation. "TPU v3." Google Cloud, 2026-10-13. docs.cloud.google.com/...v3
23. ^1 ^2 ^3 ^4Google Blog. "Our eighth generation TPUs: two chips for the agentic era." Google, 2026-04-22. blog.google/...eighth-generation-tpu-agentic-era
24. ^1 ^2 ^3 ^4 ^5Google Cloud Blog. "TPU 8t and TPU 8i technical deep dive." Google Cloud, 2026-04-23. cloud.google.com/...and-tpu-8i-technical-deep-dive
25. ^MLCommons. "MLPerf Training v0.6 results: Google systems." MLCommons, 2019-07-10. github.com/...systems
26. ^Google Cloud Blog. "Cloud TPU Pods break AI training records." Google Cloud, 2019-07-11. cloud.google.com/...pods-break-ai-training-records
27. ^MLCommons. "MLPerf Training v3.1 results: Google systems." MLCommons, 2023-11-07. github.com/...systems
28. ^MLCommons. "MLPerf Training v4.0 results: Google systems." MLCommons, 2024-06-11. github.com/...systems
29. ^MLCommons. "MLPerf Training v4.1 results: Google systems." MLCommons, 2024-11-13. github.com/...systems
30. ^Google Cloud Blog. "How to scale AI training to up to tens of thousands of Cloud TPU chips with Multislice." Google Cloud, 2023-09-01. cloud.google.com/...ltislice-to-scale-ai-workloads
31. ^1 ^2Devlin, J., et al. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." arXiv, 2018-10-11. arxiv.org/...1810.04805
32. ^Raffel, C., et al. "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer." arXiv, 2019-10-23. arxiv.org/...1910.10683
33. ^Jumper, J., et al. "Highly accurate protein structure prediction with AlphaFold." Nature, 2021-07-15. nature.com/...s41586-021-03819-2
34. ^Thoppilan, R., et al. "LaMDA: Language Models for Dialog Applications." arXiv, 2022-01-20. arxiv.org/...2201.08239
35. ^1 ^2 ^3 ^4Gemini Team, Google. "Gemini: A Family of Highly Capable Multimodal Models." arXiv, 2023-12-19. arxiv.org/...2312.11805
36. ^Gemma Team, Google DeepMind. "Gemma: Open Models Based on Gemini Research and Technology." arXiv, 2024-03-13. arxiv.org/...2403.08295
37. ^1 ^2Apple. "Apple Intelligence Foundation Language Models." arXiv, 2024-07-29. arxiv.org/...2407.21075
38. ^Google Cloud. "Cloud TPU pricing." Google Cloud, 2026-10-16. cloud.google.com/...pricing
39. ^Google Cloud Documentation. "TPU regions and zones." Google Cloud, 2026-10-13. docs.cloud.google.com/...regions-zones
40. ^Google Research. "TPU Research Cloud." Google, 2026-10-16. sites.research.google/...about
41. ^NVIDIA. "NVIDIA GB200 NVL72." NVIDIA, 2026-10-16. nvidia.com/...gb200-nvl72
42. ^Google Cloud. "Spot VMs pricing." Google Cloud, 2026-10-16. cloud.google.com/...pricing
43. ^Alphabet. "Q2 2026 Earnings Call Transcript." Alphabet Investor Relations, 2026-07-22. s206.q4cdn.com/...2026_Q2_Earnings_Transcript.pdf