Last reviewed
May 9, 2026
Sources
No citations yet
Review status
Needs citations
Revision
v3 ยท 2,977 words
Add missing citations, update stale details, or suggest a clearer explanation.
See also: Machine learning terms
TensorFlow is an open-source software library for machine learning, deep learning, and numerical computation, developed and maintained by Google. It provides end-to-end tools for building, training, deploying, and serving neural networks, running on CPUs, GPUs, and Google's purpose-built Tensor Processing Units (TPUs). TensorFlow is licensed under the Apache 2.0 license and is one of the most widely cited frameworks in the history of artificial intelligence.
This page is the gateway hub for TensorFlow-related entries on the AI Wiki. It introduces the core ideas, surveys the major APIs and surrounding ecosystem, compares TensorFlow with PyTorch and JAX, and provides a curated index of every TensorFlow concept and tool with its own dedicated wiki page.
TensorFlow grew out of an internal Google system called DistBelief, built around 2011 inside Google Brain to train large neural networks across many machines. DistBelief powered early Google products such as Inception (the image classifier behind Google Photos search) and the deep learning components of Google's speech recognition systems. Engineers including Jeff Dean, Rajat Monga, and others rewrote DistBelief into a more flexible and portable system, which Google open-sourced under the Apache 2.0 license on November 9, 2015.
The table below summarizes the major release lines.
| Version line | First release | Key change |
|---|---|---|
| TensorFlow 0.x | November 2015 | Initial open-source release. Static dataflow graphs defined in Python and run by a C++ runtime |
| TensorFlow 1.0 | February 2017 | First stable API. Introduced tf.estimator, broader Windows support, XLA experimental |
| TensorFlow 1.4 | November 2017 | Keras folded in as tf.keras. Dataset API (tf.data) added |
| TensorFlow 1.5 | January 2018 | Eager execution available as opt-in mode |
| TensorFlow 2.0 | September 2019 | Eager execution on by default. tf.function and AutoGraph for tracing. Keras became the official high-level API. Many tf.contrib modules removed |
| TensorFlow 2.x ongoing | 2020 to present | Steady additions: Keras 3, MLIR-based compiler stack, expanded TPU support, deeper JAX interoperability |
The transition from TensorFlow 1.x to 2.0 was the single largest break in the project's history. TensorFlow 1.x required users to first build a static computation graph, then execute it inside a tf.Session. TensorFlow 2.0 made eager execution the default, so code runs imperatively like ordinary Python and NumPy, and a separate tf.function decorator compiles hot paths into a graph for speed. This brought the developer experience much closer to PyTorch.
The central data structure in TensorFlow is the tensor, a multidimensional array of a single numeric or string type. A tensor has a rank (the number of dimensions), a shape (the size of each dimension), and a size (the total number of elements). All TensorFlow computations are expressed as operations on tensors.
The original TensorFlow programming model is based on a directed graph in which each node represents an operation ("op") and each edge carries a tensor from one operation's output to another's input. The runtime can then schedule the graph across multiple devices, perform automatic differentiation by traversing the graph backwards, and apply optimizations such as constant folding and operator fusion.
Key ideas inherited from this model and still present today include:
tf.GradientTape, enabling training by gradient descent and backpropagation.TensorFlow exposes a layered set of APIs. Most users today work primarily through Keras, with the lower layers reached only when needed.
Keras, originally an independent library by Francois Chollet released in 2015, became the recommended high-level API of TensorFlow in 2017 and the official one in TensorFlow 2.0. Through tf.keras, users can define models with three styles:
tf.keras.Model and write call() in pure Python.Keras provides built-in layers, losses, metrics, optimizers, and a model.fit training loop, and serializes models to SavedModel or the Keras .keras format. Keras 3, released in late 2023, is a multi-backend rewrite that runs the same model on TensorFlow, PyTorch, or JAX.
The Dataset API (tf.data) provides a high-performance input pipeline for feature engineering and data loading. It composes operations like map, batch, shuffle, cache, and prefetch into a streaming pipeline that overlaps I/O with model computation. Common sources include in-memory tensors, files in the tf.Example (TFRecord) format, CSVs, and TensorFlow Datasets (tfds).
In TensorFlow 2.x, applying @tf.function to a Python function traces the function once and produces a static graph that can be reused. AutoGraph automatically converts Python control flow (if, while, for) into graph ops. The traced graph is then compiled and optimized through the XLA (Accelerated Linear Algebra) compiler when jit_compile=True.
The tf.distribute API encapsulates strategies for parallel training across multiple devices and machines. Common strategies include:
| Strategy | Use case |
|---|---|
MirroredStrategy | Synchronous training on multiple GPUs in one machine |
MultiWorkerMirroredStrategy | Synchronous training across many machines |
TPUStrategy | Training on a TPU, TPU Pod, or TPU slice |
ParameterServerStrategy | Asynchronous training with parameter servers |
CentralStorageStrategy | Variables on CPU, computation on GPU |
The Estimator API was the recommended high-level API in TensorFlow 1.x. It encapsulated training, evaluation, prediction, and serving behind a single Estimator class, and was widely used at Google for production pipelines. With the rise of tf.keras in TensorFlow 2.x the Estimator API was placed in maintenance mode; new development is steered toward Keras, and Estimator support is being phased out of the core distribution.
Underneath Keras, TensorFlow exposes lower-level building blocks. The Layers API (tf.layers) was the canonical low-level layer API in TensorFlow 1.x and has been superseded by tf.keras.layers. tf.Module, tf.Variable, and tf.GradientTape form the minimum surface needed to write custom training loops. Queues, Savers, and summary ops are 1.x-era constructs that have been replaced by tf.data, the SavedModel format, and tf.summary writers respectively.
A defining feature of TensorFlow is its hardware reach. The runtime can dispatch the same model to several classes of accelerator.
| Hardware | Notes |
|---|---|
| CPU | Default backend. Uses Eigen and oneDNN (formerly MKL-DNN) for vectorized math |
| GPU | NVIDIA GPUs via CUDA and cuDNN. AMD GPUs via ROCm |
| TPU | Google's Tensor Processing Unit, available on Cloud TPU. A single TPU chip hosts multiple cores; many chips form a TPU Pod interconnected by a high-bandwidth fabric |
| Mobile and edge | Phones, microcontrollers, and embedded boards through TensorFlow Lite |
| Web browsers | WebGL and WebGPU through TensorFlow.js |
A TPU has its own scheduling concepts. A TPU node is a logical compute target that the user job talks to. The TPU master coordinates TPU workers, each of which owns a TPU device or several. Programs run on a TPU resource of a particular TPU type, and large jobs request a TPU slice carved out of a TPU Pod.
TensorFlow ships with or is closely accompanied by a family of supporting libraries.
TensorBoard is the visualization toolkit for TensorFlow. It reads event files written during training and displays scalar curves (loss, metric values), histograms, images, embeddings projected into 3D, computation graphs, and profiler traces. Although developed for TensorFlow, TensorBoard can also be used with PyTorch via torch.utils.tensorboard.
TensorFlow Lite is the on-device runtime. It converts a SavedModel or Keras model into a compact .tflite flatbuffer optimized for mobile (Android, iOS), embedded Linux, and microcontrollers (TFLite for Microcontrollers). It supports quantization to int8 and float16 and hardware delegates such as the Android NNAPI, the Apple Core ML delegate, and Qualcomm Hexagon. In 2024 Google began rebranding parts of TensorFlow Lite as "LiteRT" while keeping API compatibility.
TFX is the production machine learning platform layered on top of TensorFlow. It provides standard pipeline components for data ingestion, validation, preprocessing, training, evaluation, model analysis, pushing models to serving, and model card generation, orchestrated by Apache Airflow, Kubeflow Pipelines, or Apache Beam. TFX is widely used inside Google for production model lifecycles.
TensorFlow Serving is a high-performance C++ server for hosting trained models in production. It loads SavedModels, exposes them over gRPC and REST, and supports model versioning, hot reload, and batching. Serving is a key part of the path from research to deployment.
TensorFlow.js is a JavaScript library for running and training models in browsers and in Node.js. It can import a SavedModel or Keras model, execute it on the GPU through WebGL or WebGPU, and even fine-tune small models in the user's browser tab.
TensorFlow Datasets (TFDS) is a library of ready-to-use public datasets exposed as tf.data.Dataset objects, including MNIST, ImageNet, Common Crawl-derived corpora, and many more. TensorFlow Hub is a repository of pre-trained model checkpoints (image, text, audio) that can be downloaded and fine-tuned with one line of code. Many of its assets have since migrated to Kaggle Models.
TensorFlow Playground is an interactive in-browser visualization that lets users train small neural networks on toy 2D datasets and watch features form in real time. It is widely used for teaching introductory deep learning.
TensorFlow has spawned a long tail of focused libraries:
| Format | Purpose |
|---|---|
| SavedModel | Canonical export format. Bundles graph definitions, variable values, and a serving signature into a directory that any TensorFlow runtime can load |
.keras | Keras 3 unified single-file format |
HDF5 (.h5) | Legacy Keras model format |
Checkpoint (.ckpt) | Variable values written by tf.train.Checkpoint for training resume |
| TFRecord | Container of serialized tf.Example protos for efficient streaming input |
.tflite | FlatBuffer used by TensorFlow Lite |
| TensorFlow.js model JSON + binary shards | Used by TensorFlow.js |
The project's directory layout typically places SavedModels under a root directory per model, with versioned subdirectories that TensorFlow Serving can hot swap.
TensorFlow has been the implementation framework for many landmark Google models and open-source projects, including:
More recent Google frontier models such as Gemini and Gemma are primarily implemented in JAX on top of Flax.
TensorFlow's main competitors are PyTorch, released by Meta AI Research in 2016, and JAX, released by Google Research in 2018. The three frameworks make different trade-offs.
| Aspect | TensorFlow | PyTorch | JAX |
|---|---|---|---|
| Primary maintainer | Meta and the PyTorch Foundation (under the Linux Foundation since 2022) | ||
| First public release | November 2015 | September 2016 | December 2018 |
| Default execution mode | Eager since TF 2.0, optional tf.function graphs | Eager, with torch.compile graphs since 2.0 | Functional, traced with jit |
| Style | Object-oriented Keras and tf.Module | Object-oriented nn.Module | Functional, NumPy-like |
| High-level API | Keras (tf.keras) | torch.nn, Lightning, fastai | Flax, Haiku, Equinox |
| Compiler | XLA, MLIR | TorchInductor, XLA via PyTorch/XLA | XLA |
| Mobile / on-device | TensorFlow Lite, TF.js | PyTorch Mobile, ExecuTorch | Indirect (export to TFLite) |
| Production serving | TensorFlow Serving, TFX | TorchServe, third-party | Custom |
| Strongest community | Industry, Google internal | Research, open-source LLMs | Research, large-scale Google models |
For several years TensorFlow had the larger overall user base, especially in industry. From around 2019 onward, PyTorch overtook TensorFlow in the academic research community. Public analyses such as Horace He's "State of Machine Learning Frameworks" (2019) showed PyTorch share of papers at top conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR) crossing TensorFlow in 2018 to 2019, and the gap widening through 2020 and 2021. Industry use has gradually followed; most open-weights large language models released since 2022, including the LLaMA, Mistral, and Qwen families, ship with PyTorch reference implementations.
Google itself increasingly favors JAX for new frontier research, while TensorFlow remains the production workhorse for many existing Google services and a major framework for on-device and embedded inference through TensorFlow Lite.
Key inflection points in TensorFlow's adoption:
| Year | Event |
|---|---|
| 2015 | Open-source release. TensorFlow becomes the most-starred ML repo on GitHub within a year |
| 2016 to 2018 | Dominant framework in industry deep learning. Used by DeepMind, Google, Airbnb, Twitter, Uber, and many others |
| 2018 | PyTorch 1.0 stabilizes; eager execution by default attracts researchers |
| 2019 | TensorFlow 2.0 ships with eager execution and tighter Keras integration |
| 2019 to 2020 | PyTorch overtakes TensorFlow in shares of papers at top ML conferences |
| 2020 to 2022 | Hugging Face Transformers, originally TensorFlow plus PyTorch, sees PyTorch checkpoints become the default |
| 2022 to 2024 | Google increasingly publishes flagship models (PaLM 2, Gemini, Gemma) with JAX implementations |
| 2024 to 2026 | TensorFlow remains widely deployed in production and on edge devices; Keras 3 reframes Keras as multi-backend on top of TensorFlow, PyTorch, and JAX |
As of 2026, TensorFlow continues active development on a roughly quarterly release cadence, with TF 2.x as the supported line and ongoing investment in TensorFlow Lite / LiteRT, TensorFlow.js, and TPU support.
TensorFlow's enduring strengths include:
tf.distribute.TPUStrategy and Cloud TPU.Frequently cited limitations include:
tf.contrib left lingering legacy code in older tutorials.The following entries on the AI Wiki cover TensorFlow concepts, components, and surrounding hardware in more depth.