Citation and evidence

Flax (neural network library)

15 min full readUpdated 24 references

This article's verification

Report a problem with this article

Edit article
More

Use this article

Raw MarkdownExplore connections

Improve this page

Revision historyDiscuss this article

Topics

Deep LearningDeveloper ToolsGoogleOpen Source AI

Cite this article

Flax is an open-source neural network library for JAX, developed at Google and published as the Python package flax under the Apache License 2.0.[1][2][3] The JAX core package contains little that is specific to neural networks, and Flax supplies a module system, standard layers and training utilities on top of it.[24][1] It offers two APIs: Flax Linen, released in 2020 by engineers and researchers at Google Brain working with the JAX team, and Flax NNX, a simplified API released in 2024.[1][5] The project describes itself as "designed for flexibility": its documentation encourages users to try new forms of training by forking an example and changing the training loop rather than adding features to a framework.[1] The README states that Flax is maintained by a team at Google DeepMind and is not an official Google product.[1]

Linen treats a model as a pure function of its variables, in keeping with JAX's functional style. NNX instead lets models be ordinary Python objects that hold and mutate their own parameters, closer to PyTorch and Keras.[5][6][11] As of November 2026 the main documentation site covers NNX and encourages new users to adopt it, while stating that Linen will not be deprecated in the near future because most Flax users still rely on it.[4]

Overview

FieldValue
DeveloperGoogle Brain, later Google DeepMind[1][16]
Repository created2020-01-10[2]
First PyPI release0.1.0rc1, 2020-03-17[3]
Latest release (as of November 2026)0.12.12, 2026-10-27[3]
LanguagePython (requires Python 3.12 or later)[1][3]
LicenseApache License 2.0[2]
APIsflax.nnx (NNX), flax.linen (Linen)[1][9]
Documentationflax.readthedocs.io (NNX), flax-linen.readthedocs.io (Linen)[1]

Expanded article table

"Introducing Flax NNX (Part 1)", official video from Google for Developers, published December 4, 2025. Watch on YouTube.

History

Early releases and the pre-Linen API

The google/flax repository on GitHub was created on January 10, 2020, and the first release candidate of the package, 0.1.0rc1, was uploaded to the Python Package Index on March 17, 2020, followed by version 0.1.0 on April 30, 2020.[2][3] These early versions used a module API under the name flax.nn, which the changelog later calls the "pre-Linen" API.[11] Flax also shipped its own optimizers in a flax.optim package during this period.[11]

The project's design notes say that Flax was adopted by a growing set of users, "most notably in the JAX submissions for the MLPerf 0.7 benchmark", and that the team's experience over the preceding year, together with conversations with users and JAX core developers, guided a redesign of the API called Linen.[8]

Linen (2020 to 2023)

The changelog entry for version 0.3, released on PyPI on December 8, 2020, announced that "Linen is now out of Alpha" and that flax.nn was being deprecated.[11][3] Version 0.3.1 (February 2021) fully deprecated the pre-Linen classes and methods, version 0.3.6 (October 2021) moved flax.nn to flax.deprecated.nn, and version 0.4.0 (January 2022) removed it, telling users who still needed it to pin version 0.3.6.[11][3]

Optimization moved out of Flax in the same period. Version 0.4.1 (March 2022) added an upgrade guide for Optax, a separate JAX library of gradient transformations, and deprecated flax.optim; version 0.6.0 (August 2022) removed the deprecated optimizers.[11][3] Checkpointing also moved toward a separate library: the 0.9.0 release notes refer to flax.training.checkpoints as deprecated and to an Orbax upgrade guide, and current Flax releases list optax and orbax-checkpoint among their package dependencies.[12][3]

In July 2023 Google DeepMind recommended that new projects adopt Flax instead of Haiku, its own JAX neural network library. The Haiku README, which calls Flax "a neural network library originally developed by Google Brain and now by Google DeepMind", gave as reasons that Flax had a superset of Haiku's features, a larger and more active development team, more adoption outside Alphabet and more extensive documentation. Haiku entered maintenance mode, limited to bug fixes and compatibility with new JAX releases.[16]

NNX (2024 onward)

Version 0.8.0, released on January 23, 2024, added NNX, described in the changelog as "a neural network library for JAX that provides a simple yet powerful module system that adheres to standard Python semantics", with the aim of combining "the robustness of Linen with a simplified, Pythonic API akin to that of PyTorch."[11][3] The same release added nn.compact_name_scope to make porting code from Haiku easier.[11]

According to the project's "Why Flax NNX?" page, the Flax team had learned from Linen users since 2020 that some Linen ideas worked well (organizing variables into collections, automatic management of pseudorandom number generator state, and variable metadata for sharding annotations), but that Linen's lazy, functional module semantics were "non-Pythonic and often surprising", added implementation complexity and obscured the core ideas of transformations on neural networks. NNX was built to keep the useful features while introducing reference semantics into JAX.[5]

Subsequent releases changed NNX quickly:

  • 0.9.0 (August 2024) ported the Gemma transformer example to NNX, raised the minimum Python version to 3.10 and renamed the compatibility layer nnx.compat to nnx.bridge.[12][3]
  • 0.10.0 (October 2024) split the Linen and NNX documentation into separate sites, added a guide to nnx.bridge and a trimmed Linen-to-NNX migration guide.[13][3]
  • 0.11.0 (July 2025) made NNX modules JAX pytrees, reducing the need for nnx.split and nnx.merge when modules are used with plain JAX transforms, and added support for jax.experimental.MutableArray. It also included breaking changes: standard layers keep a forked copy of the random-number state passed to them, and the Optimizer no longer holds a reference to the model, which must instead be passed to update.[14][3]
  • 0.12.0 (September 2025) made nnx.Module stricter about attributes: pytree containers that hold arrays must be marked with nnx.data (or replaced by nnx.List or nnx.Dict), and attributes no longer switch automatically between data and static status. It also introduced eager sharding, in which a variable with sharding metadata is sharded on a device mesh when it is constructed.[15][3]

Releases in the 0.12 series continued through 2026, with 0.12.12 published on October 27, 2026.[3]

Release timeline

VersionDateNotable change
0.1.0rc12020-03-17First package release candidate[3]
0.1.02020-04-30First release that is not a release candidate[3]
0.3.02020-12-08Linen out of alpha; flax.nn deprecation begins[11]
0.4.02022-01-27Pre-Linen flax.deprecated.nn removed[11]
0.4.12022-03-23flax.optim deprecated in favor of Optax[11]
0.6.02022-08-17Deprecated flax.optim optimizers removed[11]
0.8.02024-01-23NNX added[11]
0.9.02024-08-27nnx.compat renamed nnx.bridge; Python 3.10 minimum[12]
0.10.02024-10-16Separate Linen and NNX documentation[13]
0.11.02025-07-29NNX modules become pytrees; Optimizer API change[14]
0.12.02025-09-25Strict pytree attributes; eager sharding[15]
0.12.122026-10-27Latest release as of November 2026[3]

Expanded article table

Dates are PyPI upload dates.[3]

Flax Linen

Functional modules

Linen is, in the words of its design notes, "fundamentally a functional system for defining neural nets in JAX."[8] Because JAX transformations such as jit, grad and vmap operate on pure functions, Linen modules are "functionalized" into explicit functions of the form

f(vin,x)→vout,yf(v_{in}, x) \rightarrow v_{out}, y

where $v_{in}$ is the variable collections and random-number state used by the model, $v_{out}$ is the mutated output variable collections, $x$ is the input data and $y$ is the output data. Applying a JAX transformation then reduces to specifying how each variable collection and random-number stream should be treated.[8]

A Linen module is a Python dataclass. Constructing one does not create any parameters; the module is "unbound" until the pure functions Module.init() or Module.apply() are called, at which point Flax clones the module and binds variables to it.[10] In a typical program, model.init(key, x) returns a dictionary of variables and model.apply(variables, x) runs the forward pass.[9]

Variables, collections and random numbers

Linen stores model data as nested dictionaries of JAX arrays organized into named collections. params is the canonical collection and generally holds the trainable weights; other collections can hold batch normalization statistics, autoregressive caches or debugging information.[10][8] The design notes say Flax deliberately keeps these kinds separate rather than grouping them as one "state", so that they can be treated differently under JAX transformations and between training and prediction. A module can also draw on several named random-number sequences, for example one for initialization and one for dropout.[8][10]

Linen initializes variables lazily: a variable is created only when the module first runs and does not find it in the supplied collections. This lets initialization and application share one code path, and it enables shape inference, in which a layer's parameter shapes are computed from the first input rather than declared in advance.[10] Submodules can be declared inline inside a single method decorated with @nn.compact, or defined in a setup() method.[10]

Lifted transformations

Because a Linen module's variables live outside the module object, applying JAX transformations inside a model requires Flax's own wrappers, called lifted transformations, such as nn.vmap and nn.scan, which annotate how each variable collection is transformed.[8][9] The "Why Flax NNX?" page lists drawbacks of this design: the lifted transforms expose APIs that are not part of JAX and can diverge from their JAX counterparts, they require a module as the first argument, they cannot return modules, and they work only inside Module.apply.[5]

Flax NNX

Modules as Python objects

In NNX a module is a regular Python object that owns its state. Parameters are created eagerly in __init__, stored as attributes wrapped in Variable types such as nnx.Param and nnx.BatchStat, and the module's methods can be called directly without a separate apply step.[6][5] There is no shape inference: input and output sizes must be given when a layer is constructed, a tradeoff the documentation describes as allowing "more explicit and predictable behavior."[5] Random numbers come from an nnx.Rngs object that produces new keys from a root seed passed to the constructor.[6]

The NNX basics guide says this design "should make PyTorch or Keras users feel at home."[6] A minimal NNX layer looks like this:[6]

from flax import nnx
import jax.numpy as jnp

class Linear(nnx.Module):
  def __init__(self, din: int, dout: int, *, rngs: nnx.Rngs):
    self.w = nnx.Param(rngs.params.uniform((din, dout)))
    self.b = nnx.Param(jnp.zeros((dout,)))

  def __call__(self, x):
    return x @ self.w + self.b[None]

model = Linear(2, 5, rngs=nnx.Rngs(params=0))
y = model(jnp.ones((1, 2)))

Because modules are mutable, stateful layers such as batch normalization update their statistics in place during the forward pass, and adding such a layer does not require changes to the training code.[5] The same property makes model surgery straightforward: a submodule can be replaced by assigning a new one, and the documentation demonstrates swapping Linear layers for custom LoRA layers this way.[5][6]

Transforms and the functional API

NNX provides its own versions of JAX transforms, including nnx.jit, nnx.value_and_grad, nnx.vmap and nnx.scan. The documentation describes them as supersets of the JAX transforms that accept modules in any argument, can return modules, can be used anywhere including the training loop, and propagate state changes made inside the transform back to the objects outside.[5][6] They can also decorate module methods, since a method is a function that takes the module as its first argument.[5] The basics guide notes that nnx.jit carries performance overhead for small models.[6]

Underneath, the NNX functional API marks the boundary between object semantics and JAX's pytree semantics. nnx.split decomposes a module into a GraphDef (static structure, analogous to JAX's PyTreeDef) and a State (the variables); nnx.merge reverses the operation; and nnx.update writes a state back into an existing object.[6] The guide explains that the functional API makes reference sharing explicit, because passing two modules that share a submodule through a JAX boundary as plain pytrees would silently lose that sharing.[6] Filters, usually Variable types, let nnx.split partition state into separate groups, for example parameters versus counters, so that only part of a model is differentiated or carried through a loop.[6]

Training typically pairs an NNX model with an Optax optimizer through nnx.Optimizer, which in current versions is updated with optimizer.update(model, grads).[4][14]

Built-in layers

The flax.nnx API includes standard layers such as Linear, Conv, BatchNorm, LayerNorm, GroupNorm, MultiHeadAttention, LSTMCell, GRUCell and Dropout.[1] The bridge guide states that all built-in Linen layers should have NNX equivalents.[7]

Linen and NNX compared

AspectFlax LinenFlax NNX
Module stateStateless; variables returned by init and held separately[7]Module owns its variables as attributes[7]
InitializationLazy, on first input, with shape inference[7][10]Eager, at construction; shapes given explicitly[7][5]
Running a modelThrough init and apply[5]Methods called directly[5]
Mutable state (for example batch statistics)Passed and returned through apply with mutable=[5]Updated in place inside the module[5]
TransformsLifted transforms with Flax-specific signatures[5]NNX transforms with JAX-style APIs[5]
Documentationflax-linen.readthedocs.io[1]flax.readthedocs.io[1]

Expanded article table

Interoperation and migration

The flax.nnx.bridge API lets one codebase mix the two styles. nnx.bridge.ToNNX wraps a Linen module so it can be used inside an NNX model or trained in an NNX loop, and nnx.bridge.ToLinen does the reverse, so a project can migrate one module at a time, top down or bottom up.[7] Linen modules wrapped for NNX keep lazy initialization through nnx.bridge.lazy_init.[7] The documentation also provides a guide for rewriting Linen code in NNX.[4]

Design philosophy

The project publishes a list of design principles. Among them: library code should be easy to read, duplicating code is preferable to a bad abstraction or to adding options to functions, base implementations should be excellent because people start projects by copying them, and an unhelpful error message is a bug.[8] The README states that the team uses GitHub for almost all of its coordination and planning, including discussion of upcoming design changes.[1]

Adoption

The Linen documentation says Flax Linen "is used by hundreds of projects", naming Hugging Face in the open-source community and Gemini, Imagen, Scenic and Big Vision at Google; its list of notable examples also includes PaLM, MaxText, T5X and Brax.[9] Several of these projects state their dependence on Flax in their own repositories:

  • T5X describes itself as a new and improved implementation of the T5 codebase, which was based on Mesh TensorFlow, in JAX and Flax.[19]
  • Big Vision, a codebase for training large-scale vision models on Cloud TPU VMs or GPUs, is based on JAX and Flax.[20]
  • Scenic, a set of libraries and projects for large-scale vision models, is developed in JAX and uses Flax.[21]
  • MaxText, an open-source LLM library and reference implementation for Cloud TPUs and GPUs, uses Flax for neural networks alongside Tunix, Orbax, Optax and Grain, and announced on June 29, 2026 that it had migrated from Linen to Flax NNX.[17]
  • Tunix, a library for post-training large language models (supervised fine-tuning and reinforcement learning), is designed to integrate with JAX modeling frameworks such as Flax NNX.[18]

Flax is also one of the packages pinned together in the JAX AI stack (jax-ai-stack), alongside JAX, Optax, Orbax, Chex, Grain and ml_dtypes. The stack's README describes these as projects Google researchers and engineers have found useful for implementing the models behind generative AI tools such as Imagen and Gemini.[24]

Hugging Face

Hugging Face's Transformers library offered Flax model implementations for several years.[22][23] In June 2021 Hugging Face announced a Flax/JAX community week, organized with Google's Flax, JAX and Cloud teams, in which each participant team was to be given free access to a TPUv3-8 virtual machine from July 7 to July 14 to build NLP and computer vision projects in JAX and Flax.[22]

In its Transformers v5 announcement on December 1, 2025, Hugging Face said it was "sunsetting our Flax/TensorFlow support in favor of focusing on PyTorch as the sole backend", while working with partners in the JAX ecosystem to keep its models compatible with that ecosystem.[23]

Continue reading

More to read

Topics:Deep LearningDeveloper ToolsGoogleOpen Source AI

References

  1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13Flax authors. "Flax: A neural network library and ecosystem for JAX designed for flexibility" (README). GitHub, google/flax, accessed 2026-11-18. github.com/...README.md
  2. ^1 ^2 ^3 ^4GitHub. "google/flax" repository metadata. GitHub, accessed 2026-11-18. github.com/...flax
  3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19Python Package Index. "flax" release history. PyPI, accessed 2026-11-18. pypi.org/...flax
  4. ^1 ^2 ^3Flax authors. "Flax". Flax documentation, accessed 2026-11-18. flax.readthedocs.io/...stable
  5. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17Flax authors. "Why Flax NNX?". Flax documentation, accessed 2026-11-18. flax.readthedocs.io/...why
  6. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Flax authors. "Flax basics". Flax documentation, accessed 2026-11-18. flax.readthedocs.io/...nnx_basics
  7. ^1 ^2 ^3 ^4 ^5 ^6 ^7Flax authors. "Use Flax NNX and Linen together". Flax documentation, accessed 2026-11-18. flax.readthedocs.io/...bridge_guide
  8. ^1 ^2 ^3 ^4 ^5 ^6 ^7Flax authors. "The Flax philosophy". Flax documentation, accessed 2026-11-18. flax.readthedocs.io/...philosophy
  9. ^1 ^2 ^3 ^4Flax authors. "Flax Linen". Flax Linen documentation, accessed 2026-11-18. flax-linen.readthedocs.io/...latest
  10. ^1 ^2 ^3 ^4 ^5 ^6Flax authors. "Glossary". Flax Linen documentation, accessed 2026-11-18. flax-linen.readthedocs.io/...glossary
  11. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13Flax authors. "CHANGELOG.md". GitHub, google/flax, accessed 2026-11-18. github.com/...CHANGELOG.md
  12. ^1 ^2 ^3Flax authors. "v0.9.0" release notes. GitHub, google/flax, 2024-08-27. github.com/...v0.9.0
  13. ^1 ^2Flax authors. "Version 0.10.0" release notes. GitHub, google/flax, 2024-10-16. github.com/...v0.10.0
  14. ^1 ^2 ^3Flax authors. "v0.11.0 - Pytrees, MutableArrays, and more!" release notes. GitHub, google/flax, 2025-07-29. github.com/...v0.11.0
  15. ^1 ^2Flax authors. "0.12.0" release notes. GitHub, google/flax, 2025-09-25. github.com/...v0.12.0
  16. ^1 ^2Google DeepMind. "Haiku: Sonnet for JAX" (README). GitHub, google-deepmind/dm-haiku, accessed 2026-11-18. github.com/...dm-haiku
  17. ^MaxText authors. "MaxText" (README). GitHub, AI-Hypercomputer/maxtext, accessed 2026-11-18. github.com/...maxtext
  18. ^Tunix authors. "Tunix: A Lightweight LLM Post-Training Library" (README). GitHub, google/tunix, accessed 2026-11-18. github.com/...tunix
  19. ^Google Research. "T5X" (README). GitHub, google-research/t5x, accessed 2026-11-18. github.com/...t5x
  20. ^Google Research. "Big Vision" (README). GitHub, google-research/big_vision, accessed 2026-11-18. github.com/...big_vision
  21. ^Google Research. "Scenic" (README). GitHub, google-research/scenic, accessed 2026-11-18. github.com/...scenic
  22. ^1 ^2Patrick von Platen. "[Open-to-the-community] Community week using JAX/Flax for NLP & CV". Hugging Face Forums, 2021-06-23. discuss.huggingface.co/...7104
  23. ^1 ^2Lysandre, Arthur Zucker, Cyril Vallez and Vaibhav Srivastav. "Transformers v5: Simple model definitions powering the AI ecosystem". Hugging Face Blog, 2025-12-01. huggingface.co/...transformers-v5
  24. ^1 ^2JAX AI stack authors. "JAX AI Stack" (README). GitHub, jax-ml/jax-ai-stack, accessed 2026-11-18. github.com/...jax-ai-stack

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

2 revisions · v2 · 2,979 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent fact-check before publication on November 18, 2026: 78 claims checked against 24 cited and independent sources; 1 material and 6 minor corrections applied before publication.

Cite this page: AI Wiki. "Flax (neural network library)." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/flax

Edit article