Machine learning terms/TensorFlow

See also: Machine learning terms

TensorFlow is an open-source software library for machine learning, deep learning, and numerical computation, developed and maintained by Google and first released to the public under the Apache 2.0 license on November 9, 2015 ^[1][19]. It provides end-to-end tools for building, training, deploying, and serving neural networks, running on CPUs, GPUs, and Google's purpose-built Tensor Processing Units (TPUs). Written primarily in Python, C++, and CUDA ^[14], TensorFlow is one of the most widely cited and most-starred frameworks in the history of artificial intelligence, with roughly 196,000 stars on GitHub as of 2026 ^[22].

This page is the gateway hub for TensorFlow-related entries on the AI Wiki. It introduces the core ideas, surveys the major APIs and surrounding ecosystem, compares TensorFlow with PyTorch and JAX, and provides a curated index of every TensorFlow concept and tool with its own dedicated wiki page.

brief history of tensorflow

TensorFlow grew out of an internal Google system called DistBelief, built around 2011 inside Google Brain to train large neural networks across many machines ^[14][19]. DistBelief powered early Google products such as Inception (the image classifier behind Google Photos search) and the deep learning components of Google's speech recognition systems. Engineers including Jeff Dean, Rajat Monga, and others rewrote DistBelief into a more flexible and portable system, which Google open-sourced under the Apache 2.0 license on November 9, 2015 ^[1][19]. Google reported that the rewritten system ran up to twice as fast as DistBelief on some benchmarks ^[19]. Adoption was rapid: at Google I/O in June 2016, Jeff Dean noted that roughly 1,500 repositories on GitHub already mentioned TensorFlow, only five of which were from Google ^[14].

The table below summarizes the major release lines.

Scroll sideways for more →

The transition from TensorFlow 1.x to 2.0, which reached general availability on September 30, 2019, was the single largest break in the project's history ^[3]. TensorFlow 1.x required users to first build a static computation graph, then execute it inside a tf.Session. TensorFlow 2.0 made eager execution the default, so code runs imperatively like ordinary Python and NumPy, and a separate tf.function decorator compiles hot paths into a graph for speed ^[4]. This brought the developer experience much closer to PyTorch ^[5].

design philosophy and core abstractions

The central data structure in TensorFlow is the tensor, a multidimensional array of a single numeric or string type. The official documentation defines tensors as "multi-dimensional arrays with a uniform type (called a dtype)" and notes that "all tensors are immutable like Python numbers and strings: you can never update the contents of a tensor, only create a new one" ^[20]. A tensor has a rank (the number of dimensions), a shape (the size of each dimension), and a size (the total number of elements). All TensorFlow computations are expressed as operations on tensors.

The original TensorFlow programming model is based on a directed graph in which each node represents an operation ("op") and each edge carries a tensor from one operation's output to another's input. As the system's designers put it, "TensorFlow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state" ^[2]. The runtime can then schedule the graph across multiple devices, perform automatic differentiation by traversing the graph backwards, and apply optimizations such as constant folding and operator fusion ^[2].

Key ideas inherited from this model and still present today include:

• Tensors as the universal value type, including dense, sparse, ragged, and string tensors.
• Operations as nodes, ranging from low-level math (matmul, conv2d) to high-level layers and optimizers.
• Graph execution for performance, alongside eager execution for ease of use.
• Automatic differentiation through tf.GradientTape, enabling training by gradient descent and backpropagation.
• Device placement, where ops can be pinned to a specific CPU, GPU, or TPU.
• Portability, with the same model definition able to run in Python for training, in C++ in production, in browsers via TensorFlow.js, and on phones via TensorFlow Lite.

what are tensorflow's core APIs?

TensorFlow exposes a layered set of APIs. Most users today work primarily through Keras, with the lower layers reached only when needed.

tf.keras

Keras, originally an independent library by Francois Chollet released in 2015 ^[18], became the recommended high-level API of TensorFlow in 2017 and the official one in TensorFlow 2.0. Through tf.keras, users can define models with three styles:

• The Sequential API, a linear stack of layers.
• The Functional API, which expresses arbitrary directed acyclic graphs of layers.
• Model subclassing, where developers extend tf.keras.Model and write call() in pure Python.

Keras provides built-in layers, losses, metrics, optimizers, and a model.fit training loop, and serializes models to SavedModel or the Keras .keras format. Keras 3, released on November 28, 2023, is a multi-backend rewrite that runs the same model on TensorFlow, PyTorch, or JAX ^[11].

tf.data

The Dataset API (tf.data) provides a high-performance input pipeline for feature engineering and data loading. It composes operations like map, batch, shuffle, cache, and prefetch into a streaming pipeline that overlaps I/O with model computation. Common sources include in-memory tensors, files in the tf.Example (TFRecord) format, CSVs, and TensorFlow Datasets (tfds).

tf.function and graph compilation

In TensorFlow 2.x, applying @tf.function to a Python function traces the function once and produces a static graph that can be reused. AutoGraph automatically converts Python control flow (if, while, for) into graph ops. The traced graph is then compiled and optimized through the XLA (Accelerated Linear Algebra) compiler when jit_compile=True ^[4].

tf.distribute

The tf.distribute API encapsulates strategies for parallel training across multiple devices and machines ^[6]. Common strategies include:

Scroll sideways for more →

estimator (deprecated)

The Estimator API was the recommended high-level API in TensorFlow 1.x. It encapsulated training, evaluation, prediction, and serving behind a single Estimator class, and was widely used at Google for production pipelines. With the rise of tf.keras in TensorFlow 2.x the Estimator API was placed in maintenance mode; new development is steered toward Keras, and Estimator support is being phased out of the core distribution ^[5].

lower-level APIs

Underneath Keras, TensorFlow exposes lower-level building blocks. The Layers API (tf.layers) was the canonical low-level layer API in TensorFlow 1.x and has been superseded by tf.keras.layers. tf.Module, tf.Variable, and tf.GradientTape form the minimum surface needed to write custom training loops. Queues, Savers, and summary ops are 1.x-era constructs that have been replaced by tf.data, the SavedModel format, and tf.summary writers respectively.

what hardware does tensorflow support?

A defining feature of TensorFlow is its hardware reach. The runtime can dispatch the same model to several classes of accelerator.

Scroll sideways for more →

Google first described the TPU's in-datacenter performance publicly in 2017 ^[17]. A TPU has its own scheduling concepts. A TPU node is a logical compute target that the user job talks to. The TPU master coordinates TPU workers, each of which owns a TPU device or several. Programs run on a TPU resource of a particular TPU type, and large jobs request a TPU slice carved out of a TPU Pod. Cloud TPUs expose these resources through Google Cloud ^[7].

the broader tensorflow ecosystem

TensorFlow ships with or is closely accompanied by a family of supporting libraries.

tensorboard

TensorBoard is the visualization toolkit for TensorFlow. It reads event files written during training and displays scalar curves (loss, metric values), histograms, images, embeddings projected into 3D, computation graphs, and profiler traces. Although developed for TensorFlow, TensorBoard can also be used with PyTorch via torch.utils.tensorboard.

tensorflow lite

TensorFlow Lite is the on-device runtime. It converts a SavedModel or Keras model into a compact .tflite flatbuffer optimized for mobile (Android, iOS), embedded Linux, and microcontrollers (TFLite for Microcontrollers). It supports quantization to int8 and float16 and hardware delegates such as the Android NNAPI, the Apple Core ML delegate, and Qualcomm Hexagon ^[8]. On September 4, 2024, Google rebranded TensorFlow Lite as LiteRT (short for "Lite Runtime") while keeping API compatibility, noting that since its 2017 debut TFLite had enabled ML features in over 100,000 apps running on 2.7 billion devices, and describing the new name as capturing a "multi-framework vision" that also supports models authored in PyTorch, JAX, and Keras ^[21].

tensorflow extended (TFX)

TFX is the production machine learning platform layered on top of TensorFlow. It provides standard pipeline components for data ingestion, validation, preprocessing, training, evaluation, model analysis, pushing models to serving, and model card generation, orchestrated by Apache Airflow, Kubeflow Pipelines, or Apache Beam ^[10]. TFX is widely used inside Google for production model lifecycles.

tensorflow serving

TensorFlow Serving is a high-performance C++ server for hosting trained models in production. It loads SavedModels, exposes them over gRPC and REST, and supports model versioning, hot reload, and batching. Serving is a key part of the path from research to deployment.

tensorflow.js

TensorFlow.js is a JavaScript library for running and training models in browsers and in Node.js. It can import a SavedModel or Keras model, execute it on the GPU through WebGL or WebGPU, and even fine-tune small models in the user's browser tab ^[9].

tensorflow datasets and tensorflow hub

TensorFlow Datasets (TFDS) is a library of ready-to-use public datasets exposed as tf.data.Dataset objects, including MNIST, ImageNet, Common Crawl-derived corpora, and many more. TensorFlow Hub is a repository of pre-trained model checkpoints (image, text, audio) that can be downloaded and fine-tuned with one line of code. Many of its assets have since migrated to Kaggle Models.

tensorflow playground

TensorFlow Playground is an interactive in-browser visualization that lets users train small neural networks on toy 2D datasets and watch features form in real time. It is widely used for teaching introductory deep learning.

specialized libraries

TensorFlow has spawned a long tail of focused libraries:

• TensorFlow Probability for probabilistic modeling and Bayesian methods.
• TensorFlow Federated for federated learning.
• TensorFlow Quantum for hybrid classical-quantum models.
• TensorFlow Recommenders for recommender systems.
• TensorFlow Graphics for differentiable rendering.
• TensorFlow Agents for reinforcement learning.
• TensorFlow Decision Forests for tree-based models.
• TensorFlow Text and KerasNLP / KerasHub for natural language processing.

file formats and serialization

Scroll sideways for more →

The project's directory layout typically places SavedModels under a root directory per model, with versioned subdirectories that TensorFlow Serving can hot swap ^[10].

major projects built with tensorflow

TensorFlow has been the implementation framework for many landmark Google models and open-source projects, including:

• The original Transformer and the paper "Attention Is All You Need" (2017), whose reference implementation was released in TensorFlow's Tensor2Tensor library ^[12].
• BERT (2018), the bidirectional encoder that briefly defined the state of the art in NLP.
• T5 (2019) and the "text-to-text" framing of language tasks.
• AlphaFold 1 (2018), DeepMind's first CASP-winning protein structure predictor, which was built on TensorFlow; the later, more accurate AlphaFold 2 (2020) was implemented in JAX and the Haiku library ^[23].
• The early generations of Google's dialogue and language models such as LaMDA and PaLM.
• TensorFlow object detection, MoveNet, MediaPipe, and many production Google features in Search, Gmail, Translate, Maps, and Photos.

More recent Google frontier models such as Gemini and Gemma are primarily implemented in JAX on top of Flax.

how does tensorflow compare with pytorch and jax?

TensorFlow's main competitors are PyTorch, released by Meta AI Research in 2016, and JAX, released by Google Research in 2018 ^[15][16]. The three frameworks make different trade-offs.

Scroll sideways for more →

For several years TensorFlow had the larger overall user base, especially in industry. From around 2019 onward, PyTorch overtook TensorFlow in the academic research community. Public analyses such as Horace He's "State of Machine Learning Frameworks" (2019) showed PyTorch share of papers at top conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR) crossing TensorFlow in 2018 to 2019, and the gap widening through 2020 and 2021 ^[13]. Industry use has gradually followed; most open-weights large language models released since 2022, including the LLaMA, Mistral, and Qwen families, ship with PyTorch reference implementations.

Google itself increasingly favors JAX for new frontier research, while TensorFlow remains the production workhorse for many existing Google services and a major framework for on-device and embedded inference through TensorFlow Lite.

adoption trajectory

Key inflection points in TensorFlow's adoption:

Scroll sideways for more →

As of 2026, TensorFlow continues active development on a roughly quarterly release cadence, with TF 2.x as the supported line and ongoing investment in TensorFlow Lite / LiteRT ^[21], TensorFlow.js, and TPU support.

what are tensorflow's strengths and limitations?

TensorFlow's enduring strengths include:

• A mature, batteries-included production stack (TFX, TensorFlow Serving, TensorFlow Lite, TensorFlow.js).
• First-class support for TPUs through tf.distribute.TPUStrategy and Cloud TPU.
• Strong tooling for visualization (TensorBoard), model analysis, fairness indicators, and model cards.
• Portability across CPU, GPU, TPU, mobile, microcontrollers, and the browser from a single source of truth.

Frequently cited limitations include:

• A historically larger and more fragmented API surface (multiple ways to do the same thing across 1.x and 2.x styles).
• A steeper learning curve for newcomers than PyTorch, particularly when crossing between eager and graph modes.
• Slower iteration in the research community compared with PyTorch and JAX.
• The deprecated Estimator API and removed tf.contrib left lingering legacy code in older tutorials.

index of tensorflow-related wiki entries

The following entries on the AI Wiki cover TensorFlow concepts, components, and surrounding hardware in more depth.

• Cloud TPU
• Dataset API (tf.data)
• device
• eager execution
• Estimator
• feature engineering
• feature spec
• graph
• graph execution
• Layers API (tf.layers)
• metric
• node (TensorFlow graph)
• operation (op)
• Parameter Server (PS)
• queue
• rank (Tensor)
• root directory
• SavedModel
• Saver
• summary
• Tensor
• TensorBoard
• TensorFlow
• TensorFlow Playground
• TensorFlow Serving
• Tensor Processing Unit (TPU)
• Tensor rank
• Tensor shape
• Tensor size
• tf.Example
• tf.keras
• TPU
• TPU chip
• TPU device
• TPU master
• TPU node
• TPU Pod
• TPU resource
• TPU slice
• TPU type
• TPU worker

see also

• TensorFlow (the framework page)
• Keras
• PyTorch
• JAX
• TensorFlow Lite
• TensorFlow.js
• XLA
• Automatic differentiation
• Google Brain
• Machine learning terms

references

1. Abadi, Martin et al. "TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems." Preliminary white paper, Google Research, November 2015. https://www.tensorflow.org/about/bib ↩
2. Abadi, Martin et al. "TensorFlow: A System for Large-Scale Machine Learning." 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 2016. https://arxiv.org/abs/1605.08695 ↩
3. TensorFlow project. "TensorFlow 2.0 is now available." TensorFlow Blog, September 30, 2019. https://blog.tensorflow.org/2019/09/tensorflow-20-is-now-available.html ↩
4. TensorFlow project. "Effective TensorFlow 2." TensorFlow Guide. https://www.tensorflow.org/guide/effective_tf2 ↩
5. TensorFlow project. "Migrate from TensorFlow 1.x to TensorFlow 2." https://www.tensorflow.org/guide/migrate ↩
6. TensorFlow project. "Distributed training with TensorFlow." https://www.tensorflow.org/guide/distributed_training ↩
7. TensorFlow project. "Cloud TPU Documentation." https://cloud.google.com/tpu/docs ↩
8. TensorFlow project. "TensorFlow Lite (LiteRT) overview." https://ai.google.dev/edge/litert ↩
9. TensorFlow project. "TensorFlow.js." https://www.tensorflow.org/js ↩
10. TensorFlow project. "TensorFlow Extended (TFX)." https://www.tensorflow.org/tfx ↩
11. TensorFlow project. "Introducing Keras 3.0." Keras Blog, November 2023. https://keras.io/keras_3/ ↩
12. Vaswani, Ashish et al. "Attention Is All You Need." NeurIPS 2017. ↩
13. He, Horace. "The State of Machine Learning Frameworks in 2019." The Gradient, October 2019. https://thegradient.pub/state-of-ml-frameworks-2019-pytorch-dominates-research-tensorflow-dominates-industry/ ↩
14. Wikipedia. "TensorFlow." https://en.wikipedia.org/wiki/TensorFlow ↩
15. Wikipedia. "PyTorch." https://en.wikipedia.org/wiki/PyTorch ↩
16. Wikipedia. "Google JAX." https://en.wikipedia.org/wiki/Google_JAX ↩
17. Jouppi, Norman P. et al. "In-Datacenter Performance Analysis of a Tensor Processing Unit." ISCA 2017. ↩
18. Chollet, Francois. *Deep Learning with Python*, 2nd edition. Manning, 2021. ↩
19. Google Research Blog. "TensorFlow: Google's latest machine learning system, open sourced for everyone." November 9, 2015. https://research.google/blog/tensorflow-googles-latest-machine-learning-system-open-sourced-for-everyone/ ↩
20. TensorFlow project. "Introduction to Tensors." TensorFlow Guide. https://www.tensorflow.org/guide/tensor ↩
21. Google Developers Blog. "TensorFlow Lite is now LiteRT." September 4, 2024. https://developers.googleblog.com/en/tensorflow-lite-is-now-litert/ ↩
22. TensorFlow. "tensorflow/tensorflow: An Open Source Machine Learning Framework for Everyone." GitHub repository. https://github.com/tensorflow/tensorflow ↩
23. Google DeepMind. "AlphaFold." GitHub repository (open source code for AlphaFold 2, implemented in JAX and Haiku). https://github.com/google-deepmind/alphafold ↩