Citation and evidence

torch.compile

21 min full readUpdated 25 references

This article's verification

Report a problem with this article

Edit article
More

Use this article

Raw MarkdownExplore connections

Improve this page

Revision historyDiscuss this article

Topics

Deep LearningDeveloper ToolsTraining & Optimization

Cite this article

torch.compile is the just-in-time graph capture and compilation feature introduced in PyTorch 2.0, a release first announced at the PyTorch Conference on December 2, 2022 and shipped as a stable version on March 15, 2023.[1][2] Wrapping a model with a single call (torch.compile(model)) routes its Python execution through a stack of new components, TorchDynamo, AOTAutograd, PrimTorch, and TorchInductor, that lift unmodified eager-mode code into optimized Triton kernels on NVIDIA A100-class GPUs or C++/OpenMP code on CPUs.[1][3] Meta and the PyTorch Foundation reported weighted-average training speedups of 38% on TIMM, 52% on HuggingFace Transformers, and 76% on TorchBench at the December 2022 announcement, with a follow-on ASPLOS 2024 paper measuring 2.27x inference and 1.41x training speedup across 180+ models on an A100.[1][3] The design preserves PyTorch's "eager by default" programming model while introducing optional ahead-of-time compilation, which the PyTorch team called "a substantial new direction for PyTorch" and the reason for the 2.0 version number.[1]

Infobox

FieldValue
Feature nametorch.compile
ProjectPyTorch 2.x
DeveloperMeta AI, PyTorch Foundation, broader open-source contributors
First announcedDecember 2, 2022 (PyTorch Conference)
Stable releaseMarch 15, 2023 (PyTorch 2.0)[2]
LicenseBSD-3-Clause (PyTorch repository)
Default backendTorchInductor
GPU codegen targetOpenAI Triton
CPU codegen targetC++ with OpenMP
Repositorygithub.com/pytorch/pytorch
Reference paperAnsel et al., ASPLOS 2024[3]

Expanded article table

"torch.compile: The Missing Manual", official video from PyTorch, published August 13, 2024. Watch on YouTube.

Background

PyTorch was originally a "define-by-run" framework: each tensor operation executes immediately in the Python interpreter, which is the eager mode that made the library popular among researchers but historically slower than graph-based frameworks like TensorFlow 1 or XLA-backed systems.[4] Several earlier attempts to add compilation to PyTorch (TorchScript, FX tracing, Lazy Tensor, and various nvFuser/NNC fusers) each captured only part of a model or required users to rewrite code, and none became the default path.[3]

Jason Ansel started the TorchDynamo and TorchInductor projects at Meta.[5] The PyTorch 2.0 announcement says the team had spent the previous five years building earlier capture systems (torch.jit.trace, TorchScript, FX tracing, and Lazy Tensors) without finding one that gave them everything they wanted.[1] The official PyTorch 2.0 retrospective frames TorchDynamo's bytecode-rewriting approach as "a result of 5 years of our R&D into safe graph capture."[1]

The public roll-out followed a deliberate sequence:

  • December 2, 2022. PyTorch 2.0 unveiled at the PyTorch Conference in New Orleans, with torch.compile available in nightly builds and a stable release promised for early March 2023.[1][6]
  • March 15, 2023. PyTorch 2.0 released as a stable version, accompanied by the official blog post "PyTorch 2.0: Our next generation release that is faster, more Pythonic and Dynamic as ever."[2]
  • October 2023. PyTorch 2.1 ships at the PyTorch Conference 2023, adding automatic dynamic shape support to torch.compile, NumPy API support inside compiled regions, AVX-512 support in the CPU Inductor backend, an enhanced SDPA implementation, and a prototype of torch.export.[18]
  • January 30, 2024. PyTorch 2.2 ships with FlashAttention-2 integration in scaled_dot_product_attention, the AOTInductor ahead-of-time deployment path, optimizer compilation improvements, horizontal fusion for torch.cat, and the unified TORCH_LOGS logging system.[19]
  • April 27 to May 1, 2024. The 49-author paper "PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation" presented at the 29th ACM ASPLOS conference in San Diego, with the proceedings published in early 2024.[3][5]

PyTorch 2.0 is fully backward-compatible with PyTorch 1.x: torch.compile is an opt-in wrapper, not a new programming model, and code that does not call it continues to execute eagerly with no changes.[1][2] The PyTorch Foundation, the Linux Foundation-hosted body that took stewardship of the project in late 2022, presents torch.compile as the flagship feature of the 2.x line and the central piece around which subsequent releases organize their performance work.[4][19]

How It Works

torch.compile(fn) returns a callable that, on first invocation, traces and compiles the underlying Python function or nn.Module. The stack underneath is layered, with each component responsible for a different stage of lowering a Python program down to device code.[1][3]

TorchDynamo (graph capture)

TorchDynamo is a Python-level just-in-time compiler that hooks into the CPython frame evaluation API specified by PEP 523.[7][8] When a wrapped function is called, Dynamo intercepts the frame, walks the function's bytecode, and rewrites it on the fly: PyTorch operations are extracted into an FX graph (torch.fx.GraphModule), while constructs that cannot be safely captured (data-dependent control flow, arbitrary Python side effects, calls to C extensions) cause a "graph break" and fall back to the regular CPython interpreter.[3][9]

Because Dynamo operates on bytecode rather than the Python source, it can capture code paths that traditional symbolic tracing cannot reach, including if/else branches, list comprehensions, and many third-party library calls. The ASPLOS 2024 paper reports that Dynamo captures graphs more robustly than prior PyTorch approaches while adding minimal overhead.[3] On a 7,000-plus repository GitHub corpus, Meta reports a 99% graph-capture rate, with the remaining 1% falling back to eager execution.[1]

PEP 523, accepted into Python 3.6, added a hook in the CPython interpreter that lets external C code substitute its own frame-evaluation function in place of _PyEval_EvalFrameDefault. Originally motivated as a generic JIT entry point, it became the foundation TorchDynamo uses to intercept Python execution without modifying the user's source code or requiring AST-level analysis.[7] When TorchDynamo is active it installs a custom frame evaluator that, for each Python function call, decides whether to rewrite the bytecode (extracting an FX graph and substituting calls to the compiled artifact) or to defer to the default evaluator. Cached compilations are keyed on the bytecode object plus the guard set, so successive calls with the same shapes and types reuse the existing compiled artifact at near-zero overhead.[7][9]

Guard-based recompilation

Each compiled graph is paired with a set of "guards", runtime checks that confirm the assumptions Dynamo made during tracing (tensor shapes, dtypes, attribute values, Python types) still hold for a new call.[9][10] If all guards pass the cached compiled artifact is reused; if a guard fails, Dynamo retraces and produces a new specialization, optionally widening the assumption (for example, converting a fixed shape into a symbolic one) so the next call is more likely to hit the cache.[9]

By default dynamic=None, automatic dynamic shapes: the first call specializes on observed shapes, and if subsequent calls violate that guard the function is recompiled with symbolic shape support, using a ShapeEnv and SymPy expressions to reason about size relationships.[10] Setting dynamic=True skips the first specialization and traces symbolically from the start. Setting TORCH_LOGS=recompiles causes the compiler to log each recompilation along with the guards that triggered it, which is the primary debugging tool for guard failures.[10]

AOTAutograd (joint forward/backward capture)

To accelerate training and not just inference, PyTorch 2.0 introduced AOTAutograd, an "ahead-of-time" autograd engine that traces both the forward and backward graphs by re-using the standard dispatcher.[1][3] AOTAutograd intercepts the captured Dynamo graph, decomposes operators against the PyTorch dispatcher, runs the autograd engine ahead of time on FakeTensors, and emits a joint forward-and-backward FX graph that the downstream compiler can optimize as a single unit.[3] Without AOTAutograd, only the forward pass would be compiled, leaving training largely on the eager path; with it, the entire training step (including activation save/restore) becomes a single compiled artifact.[1]

PrimTorch (operator decomposition)

PyTorch has on the order of two thousand operators (counting overloads), too many for any single backend to implement directly. PrimTorch defines two smaller, stable operator sets that the captured graph is lowered into: the "Prim ops" set of roughly 250 low-level primitives suited for compilers, and the "ATen ops" set of approximately 750 higher-level operators suited for backends that prefer to consume larger building blocks.[1] Decompositions are written in Python so they can be reused across backends and traced through Dynamo.[1][3]

TorchInductor (default code generator)

TorchInductor is the default backend that consumes the post-AOTAutograd, post-PrimTorch graph and produces device code.[1][3] On NVIDIA GPUs it lowers to OpenAI Triton kernels; on CPUs it generates C++ source files compiled with a normal host compiler and parallelized with OpenMP.[3] Inductor uses a Python loop-level intermediate representation with about 50 operators that is deliberately small and extensible, and it performs scheduling, fusion, and memory-planning passes before emitting kernels.[1] The ASPLOS 2024 paper reports that Inductor outperforms six other compilers tested in the same harness across 180+ models.[3]

Code generation proceeds in three logical phases. First, the captured FX graph is decomposed and normalized using PrimTorch decompositions; second, Inductor lowers the result into its loop-level IR and performs scheduling decisions (op fusion, tiling, memory layout); third, it emits Triton kernels (for CUDA and ROCm targets) or C++ source files (for CPU targets) and compiles them with the corresponding toolchain. The Triton path benefits from Triton's block-level programming model, which lets a single kernel author cover a wide range of tile sizes and broadcasting patterns without writing per-shape variants.[1][3]

Compile modes

torch.compile accepts a mode argument that selects an optimization preset:

ModeBehaviorTypical use
"default"Balanced compile time and runtime, suitable for large models.Most training and inference workloads.[11]
"reduce-overhead"Wraps the compiled graph in CUDA Graphs to eliminate per-launch Python and driver overhead, at the cost of additional memory.Small models and low-latency inference where launch overhead dominates.[11][12]
"max-autotune"Searches over Triton kernel variants, fusion schedules, and templated GEMMs (with optional Cutlass and cuDNN templates); compile times are much longer but generated code is the fastest available.Latency-critical deployment or repeated training runs where compile cost amortizes.[13]

Expanded article table

Additional keyword arguments include fullgraph=True (raise on any graph break, useful for export and debugging) and dynamic=True/False/None (control symbolic-shape behavior).[9][10] The reduce-overhead mode is implemented via "CUDAGraph Trees", a refinement that records multiple captured graphs sharing a single memory pool so that different execution paths can be replayed without separate allocations.[12]

CUDA Graph Trees specifically address a problem that arises when chaining several captured CUDA graphs together: a naive approach would force each graph to use a separate memory pool, blowing up activation memory and adding host-side copies to move intermediates between graphs. Trees instead share a single pool across all graphs in the tree and use a tensor-liveness tracker (implemented in torch/_inductor/cudagraph_trees.py) to reuse dead memory across replays, retaining the launch-overhead savings without paying the memory cost.[12][22] Because CUDA Graphs require fixed tensor addresses and shapes, Trees re-record a fresh CUDAGraph for each unique input shape; for workloads with very high shape variability, the re-record cost can outweigh the savings, in which case mode="default" (without CUDA Graphs) is preferred.[12][22]

torch.export and AOTInductor

torch.export is a sibling API that reuses Dynamo to capture a "complete" graph (one with no graph breaks) of an nn.Module plus example inputs, then serializes it as an ExportedProgram. PyTorch 2.2 added AOTInductor, an ahead-of-time variant of TorchInductor that consumes an exported program and emits a compiled shared library suitable for loading in non-Python server-side environments, sharing the same Triton/C++ codegen as the JIT path.[19] This lets a single backend serve both interactive development (via torch.compile) and production deployment (via torch.export and AOTInductor).

Implementations and Backends

Although TorchInductor is the default, torch.compile exposes a backend= argument that selects an alternative compiler; the canonical list can be retrieved with torch.compiler.list_backends(). In late 2023 it returned cudagraphs, inductor, onnxrt, openxla, openxla_eval, and tvm.[15] The onnxrt backend was removed from core in July 2025, so it is absent from PyTorch 2.9 onward. In PyTorch 2.15 the in-tree backends are inductor, cudagraphs, openxla, and tvm, with openxla_eval registered as experimental.[23][24] Custom backends can be registered through the torch._dynamo extension API; the developer documentation walks through the contract a backend must satisfy (it receives an FX GraphModule and a list of example inputs, and returns a callable).[15]

Notable third-party backends include:

BackendMaintainerTargetNotes
TorchInductorMeta / PyTorchNVIDIA GPUs (Triton), AMD GPUs (Triton on ROCm), CPUs (C++/OpenMP)Default backend; the focus of the ASPLOS 2024 paper.[3]
cudagraphsPyTorchNVIDIA GPUsWraps captured graph in CUDA Graphs without Inductor codegen.[12]
onnxrtMicrosoft / PyTorchONNX Runtime targetsLowered FX graphs to ONNX and dispatched via ONNX Runtime; removed from core in July 2025 (absent from PyTorch 2.9 onward).[14][23]
openxlaGoogle / PyTorchXLA-supported hardware (TPUs, GPUs)Routes through XLA via the PyTorch/XLA integration.[14]
tvmApache TVMMultipleLowers through Apache TVM's compiler stack.[14]
HidetCentML, University of Toronto, AWSNVIDIA GPUsSelected via backend="hidet"; aimed at inference with operator-schedule search and Tensor Core utilization.[16]
Intel Extension for PyTorch (IPEX)Intel (OpenVINO ecosystem)Intel CPUs and Intel GPUsProvides Intel-optimized fusion patterns and dispatches to Intel libraries.[14]

Expanded article table

A September 2023 RFC proposed moving most of these backends out of torch core by December 2023. Some, such as IPEX and nvFuser, were removed, but the RFC was closed as not planned in March 2024, and the in-tree set still includes Inductor, CUDA Graphs, and a few cross-vendor adapters.[14][24]

Performance Numbers

Two distinct sets of headline numbers are commonly cited.

The PyTorch Conference 2022 keynote (December 2, 2022) and the official PyTorch 2.0 announcement reported, across 163 diverse open-source models tested on an A100:[1]

Benchmark suiteModels testedTraining speedup (weighted)
HuggingFace Transformers4652%
TIMM (PyTorch Image Models)6138%
TorchBench5676%
Overall16343% (21% FP32, 51% AMP)

Expanded article table

torch.compile was reported to work on 93% of the 163 models tested, with the weighted average defined as 0.75 times the AMP speedup plus 0.25 times the FP32 speedup to reflect that automatic mixed precision is the more common training configuration.[1]

The ASPLOS 2024 paper, which uses a refreshed harness covering 180+ models on the same A100 hardware, reports a geometric-mean inference speedup of 2.27x and a geometric-mean training speedup of 1.41x for TorchInductor over eager mode, and finds that TorchInductor outperforms each of six other compared compilers across the harness.[3] The paper also evaluates Dynamo's graph-capture coverage and overhead and presents an ablation of TorchInductor optimizations such as fusion, inlining, loop and layout reordering, matmul templates, pattern matching, and CUDA Graphs.[3]

PyTorch maintains a public nightly performance dashboard that runs the same HuggingFace, TIMM, and TorchBench harnesses on 12 GCP A100 nodes (each with a 40 GB A100) and publishes speedup, memory, and pass-rate metrics for every nightly build.[17]

A second category of results comes from end-application studies. The November 2023 PyTorch blog "PyTorch compile to speed up inference on Llama 2" reports the following inference latencies, measured on A100 80 GB hardware and with torch.compile plus SDPA (and tensor parallelism for the 70B configuration) compared against unoptimized eager execution:[20]

ModelGPUsLatency with torch.compile (ms/token)Speedup vs. unoptimized
Llama 2 7B1 x A100reported sub-linear scaling vs. sequence lengthreported "significant" speedup[20]
Llama 2 13B1 x A100reported sub-linear scaling vs. batch sizereported "significant" speedup[20]
Llama 2 70B8 x A100292.4x[20]

Expanded article table

For inference of compute-bound diffusion and transformer pipelines, the Hugging Face Diffusers PyTorch 2.0 documentation reports that torch.compile can provide "an additional speed-up of 5-300x on top of SDPA" depending on GPU architecture, with Ampere (A100, 3090), Ada (4090), and Hopper (H100) showing the largest gains. The benchmark tables on the same page show much smaller gains, for example 21.66 to 44.03 iterations per second (about 2x) for Stable Diffusion text-to-image on an A100 at batch size 1, so the "5-300x" wording is not borne out by the page's own measurements.[21]

Applications

torch.compile is now the recommended path for accelerating training and inference of standard PyTorch models without rewriting them. Concrete deployment patterns include:

  • Speeding up Transformer training in Hugging Face Transformers and similar libraries by adding a single model = torch.compile(model) line after model construction. The PyTorch 2.0 announcement cites Sylvain Gugger reporting a 1.5x to 2x speedup on Hugging Face training scripts with no other changes.[1]
  • Accelerating computer-vision training: TIMM models pass through torch.compile largely without modification, per Ross Wightman's quoted endorsement in the PyTorch 2.0 announcement.[1]
  • Reducing inference latency on small models by combining mode="reduce-overhead" with CUDA Graphs to amortize kernel-launch and Python-interpreter overhead, which dominates eager-mode execution for short generation steps in language models.[11][12]
  • Running models on other runtimes via alternate backends, for example backend="openxla" for XLA-targeted hardware (a backend="onnxrt" option for ONNX Runtime existed through PyTorch 2.8).[14][23]
  • Distributed training: torch.compile is integrated with both DistributedDataParallel and FullyShardedDataParallel; the announcement reports up to 15% (FP32) and 80% (AMP) gains for compiled DDP.[1]
  • Large-language-model inference: a November 2023 PyTorch blog post from an IBM Research and Meta team reports that combining torch.compile, SDPA (with FlashAttention), and tensor parallelism yields 29 ms/token latency on a 70-billion-parameter Llama 2 model running on 8 A100 80 GB GPUs, a 2.4x improvement over unoptimized inference under the same conditions (512-token input, 50-token generation).[20]
  • High-throughput attention: from PyTorch 2.2 onward, scaled_dot_product_attention automatically dispatches to FlashAttention-2 on supported hardware, achieving 50 to 73 percent of theoretical maximum FLOPs on A100; combining SDPA with torch.compile is the standard recipe in current PyTorch examples.[19]

It also unlocks ahead-of-time export. The torch.export API leverages Dynamo's graph capture to produce a serializable program representation, and AOTInductor (introduced in PyTorch 2.2) takes that exported graph and compiles it into a shared library suitable for non-Python server-side deployments, sharing backend code with the JIT path.[19]

Limitations and Criticisms

Despite the speedups, the system has known constraints and rough edges:

  • Coverage is high but not 100%. The original announcement places coverage at 93% of 163 tested models, and graph breaks (where Dynamo cannot proceed and falls back to the interpreter) reduce the size of the compiled region; in extreme cases, performance can match or trail eager mode.[1][9]
  • Cold-start compile times are non-trivial, particularly for mode="max-autotune", which searches over Triton kernel variants and can take significantly longer to first-call than mode="default". The PyTorch max-autotune tutorial describes the mode as trading longer compilation times for improved runtime performance, so it pays off only when the compiled model runs often enough to amortize that cost.[13]
  • Guard misses cause recompilation. Programs that pass tensors of many different shapes (variable-length sequences, ragged batches) may recompile repeatedly unless dynamic=True is enabled or shape ranges are constrained. Each recompilation adds latency and produces a distinct cache entry.[9][10]
  • Hardware coverage at launch was narrow. The PyTorch 2.0 announcement explicitly noted that only NVIDIA Volta and Ampere GPUs and CPUs were supported out of the box, with speedups on desktop-class GPUs such as the NVIDIA 3090 lower than on the A100; broader GPU and accelerator coverage arrived in subsequent 2.x releases.[1]
  • Debugging is harder than eager. Stack traces span Dynamo-rewritten bytecode, AOTAutograd-generated graphs, and Triton-emitted code, requiring developers to learn new logs (TORCH_LOGS=recompiles,graph_breaks,dynamo) and tools (torch._dynamo.explain) to localize regressions.[9][10]
  • max-autotune on CPU is less mature than on CUDA, with only a subset of operations using templated kernels at launch and ongoing work to extend the GEMM template coverage.[13]

Version Timeline

Subsequent PyTorch 2.x releases have continued to expand torch.compile rather than supersede it; as of November 2026 the latest release is PyTorch 2.15.0, published October 28, 2026.[25] Notable changes in the early 2.x releases include:

VersionReleasedKey torch.compile additions
2.02023-03-15Initial public release of torch.compile with TorchDynamo, AOTAutograd, PrimTorch, and TorchInductor; modes default, reduce-overhead, max-autotune; dynamic, fullgraph, backend parameters; SDPA front-end with FlashAttention v1 / Memory-Efficient Attention.[2]
2.1October 2023Automatic dynamic shape support; NumPy API support inside compiled regions; CPU Inductor AVX-512 codegen; improved SDPA pattern matching.[18]
2.22024-01-30FlashAttention-2 integration; AOTInductor (ahead-of-time deployment); compiled optimizer improvements; horizontal fusion for torch.cat; TORCH_LOGS standardization.[19]

Expanded article table

The PyTorch Foundation publishes per-release notes via the project blog and the GitHub release tags, and the developer mailing list dev-discuss.pytorch.org tracks ongoing torch.compile RFCs.[4][19]

Comparison

torch.compile is one of several frameworks-level graph compilers active in the deep-learning ecosystem.

SystemCapture mechanismDefault codegenEager supportSponsor
torch.compile (PyTorch 2)Python bytecode rewrite via PEP 523TorchInductor → Triton / C++Yes, opt-in JITMeta / PyTorch Foundation[1][3]
TorchScript (PyTorch 1)Source tracing / scriptingTorchScript IR + custom fusersYes (separate path)Meta
XLA (TensorFlow, JAX)Lazy tensor tracing or jit decoratorXLA HLO → backendTF eager, JAX trace-on-callGoogle
Apache TVM (tvm backend)Relay/Relax IR importTVM tensor IRStandalone compilerApache, OctoML
ONNX Runtime (onnxrt backend)ONNX graph importORT execution providersStandalone runtimeMicrosoft, ONNX consortium

Expanded article table

Compared to TorchScript, torch.compile distinguishes itself by not requiring users to make their models scriptable: Dynamo's bytecode-level capture handles native Python control flow that TorchScript could not. Compared to JAX's jit, PyTorch keeps eager mode as the default and treats compile as an optional wrapper rather than the dominant API.[1][3]

A second axis of comparison is the codegen target. Where XLA and TVM use their own internal intermediate representations and target a wide range of hardware through compiler back-ends, TorchInductor stands out by emitting source code in two well-known languages: Triton (a Python-embedded DSL designed at OpenAI for GPU kernel programming) and standard C++ with OpenMP. The Python-implemented Inductor IR is intentionally small (around 50 operators), which the ASPLOS 2024 paper argues lets compiler engineers prototype optimizations in Python and avoids the upfront engineering cost of a custom IR per backend.[1][3]

  • PyTorch, the framework that hosts torch.compile.
  • Triton (compiler), the GPU kernel language TorchInductor emits.
  • JAX, a contemporary framework with a different graph-capture model (jit).
  • TensorFlow and its XLA backend, the major prior-generation graph compiler.
  • ONNX and the onnxrt backend for cross-framework deployment.
  • OpenVINO and Intel Extension for PyTorch, an Intel-targeted backend lineage.
  • NVIDIA A100, the GPU on which the headline benchmarks were collected.
  • Python PEP 523, the CPython frame evaluation API TorchDynamo uses.
  • FX, the PyTorch IR that TorchDynamo emits and TorchInductor consumes.

Continue reading

More from See also

Topics:Deep LearningDeveloper ToolsTraining & Optimization

References

  1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27PyTorch Team, "PyTorch 2.x: faster, more pythonic and dynamic as ever", PyTorch, 2022-12-02 (updated for the 2.0 release). pytorch.org/...pytorch-2-x. Accessed 2026-05-21.
  2. ^1 ^2 ^3 ^4 ^5PyTorch Foundation, "PyTorch 2.0: Our next generation release that is faster, more Pythonic and Dynamic as ever", PyTorch Blog, 2023-03-15. pytorch.org/...pytorch-2-0-release. Accessed 2026-11-05.
  3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21Jason Ansel, Edward Yang, Horace He, et al., "PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation", Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '24), 2024-04-27. docs.pytorch.org/...pytorch2-2.pdf. Accessed 2026-05-21.
  4. ^1 ^2 ^3PyTorch Team, "PyTorch 2 paper and tutorial @ ASPLOS 2024", PyTorch Blog, 2024-04. pytorch.org/...pytorch-pytorch-2-paper-tutorial. Accessed 2026-05-21.
  5. ^1 ^2Jason Ansel, "About", jasonansel.com, accessed 2026. jasonansel.com. Accessed 2026-05-21.
  6. ^PyTorch (official account), "We just introduced PyTorch 2.0 at the #PyTorchConference, introducing torch.compile! Available in the nightlies today, stable release Early March 2023", X (Twitter), 2022-12-02. x.com/...1598708792598069249. Accessed 2026-05-21.
  7. ^1 ^2 ^3Brett Cannon and Dino Viehland, "PEP 523: Adding a frame evaluation API to CPython", Python Software Foundation, 2016. peps.python.org/pep-0523. Accessed 2026-05-21.
  8. ^PyTorch contributors, "torchdynamo (project README)", PyPI, 2022. pypi.org/...torchdynamo. Accessed 2026-05-21.
  9. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8PyTorch Team, "Dynamo Overview", PyTorch documentation. docs.pytorch.org/...torch.compiler_dynamo_overview. Accessed 2026-05-21.
  10. ^1 ^2 ^3 ^4 ^5 ^6PyTorch Team, "Dynamic Shapes Core Concepts", PyTorch documentation. docs.pytorch.org/...dynamic_shapes_core_concepts. Accessed 2026-05-21.
  11. ^1 ^2 ^3PyTorch Team, "Introduction to torch.compile", PyTorch Tutorials. docs.pytorch.org/...torch_compile_tutorial. Accessed 2026-05-21.
  12. ^1 ^2 ^3 ^4 ^5 ^6PyTorch Team, "CUDAGraph Trees", PyTorch documentation. docs.pytorch.org/...torch.compiler_cudagraph_trees. Accessed 2026-05-21.
  13. ^1 ^2 ^3PyTorch Team, "Using Max-Autotune Compilation on CPU for Better Performance", PyTorch Tutorials. docs.pytorch.org/...max_autotune_on_CPU_tutorial. Accessed 2026-05-21.
  14. ^1 ^2 ^3 ^4 ^5 ^6PyTorch contributors, "[RFC]: Moving most torch.compile backends out of core", pytorch/pytorch issue #109687, GitHub, 2023-09. github.com/...109687. Accessed 2026-05-21.
  15. ^1 ^2PyTorch Forums, "Where to begin developing custom backend for torch.compiler?", discuss.pytorch.org. discuss.pytorch.org/...191182. Accessed 2026-05-21.
  16. ^Team Hidet, "Introducing Hidet: A Deep Learning Compiler for Efficient Model Serving", PyTorch Blog, 2023-04-27. pytorch.org/...introducing-hidet. Accessed 2026-05-21.
  17. ^PyTorch Team, "PyTorch 2.0 Performance Dashboard", PyTorch documentation. docs.pytorch.org/...compiler_performance_dashboard. Accessed 2026-05-21.
  18. ^1 ^2Andrew Hoblitzell, "PyTorch 2.1 Release Supports Automatic Dynamic Shape Support and Distributed Training Enhancements", InfoQ, 2023-10. infoq.com/...pytorch21-at-pytorch-con-2023. Accessed 2026-05-21.
  19. ^1 ^2 ^3 ^4 ^5 ^6 ^7PyTorch Team, "PyTorch 2.2: FlashAttention-v2 integration, AOTInductor", PyTorch Blog, 2024-01-30. pytorch.org/...pytorch2-2. Accessed 2026-05-21.
  20. ^1 ^2 ^3 ^4 ^5Antoni Viros i Martin, Brian Vaughan, Davis Wertheimer, Joshua Rosenkranz, Mudhakar Srivatsa, Nelson Mimura Gonzalez, Raghu Ganti, Supriyo Chakraborty, Zhuoran Liu, Geeta Chauhan, Hamid Shojanazeri, "PyTorch compile to speed up inference on Llama 2", PyTorch Blog, 2023-11-07. pytorch.org/...pytorch-compile-to-speed-up-inference. Accessed 2026-05-21.
  21. ^Hugging Face, "PyTorch 2.0", Diffusers documentation (v0.25.0). huggingface.co/...torch2.0. Accessed 2026-11-05.
  22. ^1 ^2PyTorch contributors, "torch/_inductor/cudagraph_trees.py", pytorch/pytorch source tree, GitHub. github.com/...cudagraph_trees.py. Accessed 2026-05-21.
  23. ^1 ^2 ^3PyTorch contributors, "[ONNX] Remove legacy Dort (#158258)", pytorch/pytorch commit 5606c516fd, GitHub, 2025-07-15. github.com/...c516fd87e5c3594177e4ca64c3cac7fdafd5. Accessed 2026-11-05.
  24. ^1 ^2PyTorch contributors, "torch/_dynamo/backends" (v2.15.0), pytorch/pytorch source tree, GitHub. github.com/...backends. Accessed 2026-11-05.
  25. ^PyTorch contributors, "Releases", pytorch/pytorch, GitHub. github.com/...releases. Accessed 2026-11-05.

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

6 revisions by 1 contributor · v6 · 4,281 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent fact-check of the full article against its cited sources (95 claims, 30 sources fetched); 4 material + 17 minor fixed

Cite this page: AI Wiki. "torch.compile." aiwiki.ai, updated 21 May 2026, fact-checked 5 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/torch_compile

Edit article