Apple Neural Engine

The Apple Neural Engine (ANE) is the dedicated neural network accelerator, a type of neural processing unit (NPU), that Apple builds into the Apple silicon system-on-chip powering iPhone, iPad, Mac, Apple Watch, Apple Vision Pro, and HomePod. It first shipped in the A11 Bionic chip inside iPhone 8, iPhone 8 Plus, and iPhone X on September 12, 2017 as a two-core unit rated at 600 billion operations per second (0.6 TOPS), and by the M4 in 2024 it had grown to a 16-core design rated at 38 trillion operations per second (38 TOPS), which Apple says is 60 times faster than the original ^[1][2][21]. The ANE accelerates on-device machine learning inference, such as Face ID, computational photography, speech recognition, and, since 2024, the on-device large language model that underpins Apple Intelligence, delivering roughly an order of magnitude better performance per watt than running the same models on the CPU or GPU ^[3][4][6].

Developers do not target the ANE directly. Instead, they ship Core ML models built with the Python coremltools library, and the Core ML runtime decides at load time which subgraphs of the model to dispatch to the ANE, the GPU, or the CPU based on the operations supported, the precision required, and the device's current thermal and power state ^[5]. With the launch of Apple Intelligence at WWDC on June 10, 2024, the ANE became the execution target for a roughly three billion parameter on-device foundation model that powers writing tools, notification summaries, the rebuilt Siri, Genmoji, and Image Playground on supported iPhone, iPad, and Mac hardware ^[6][7].

What is the Apple Neural Engine?

The ANE, sometimes referred to internally as the Neural Processing Unit, is a fixed-function block on the same SoC die as the CPU, GPU, image signal processor, and Secure Enclave. It exists for one reason: to run neural network inference far more efficiently than a general-purpose CPU or GPU. Apple's neural accelerator program traces back to a small silicon team that began work on a custom inference engine for Face ID in the mid-2010s. The first hardware shipped to consumers in late 2017 and has been iterated every year, with the macro story being a steady increase in core count, throughput, and supported numerical precisions, plus closer integration with transformer workloads from 2022 onward ^[3][8].

How fast is the Apple Neural Engine?

The ANE has roughly doubled in throughput at each major step, growing from 0.6 TOPS on the A11 in 2017 to 35 TOPS on the A17 Pro and 38 TOPS on the M4 in 2024. When Apple introduced the M4, it described the part in plain superlatives: "M4 has Apple's fastest Neural Engine ever, capable of up to 38 trillion operations per second, which is faster than the neural processing unit of any AI PC" ^[21]. Apple frames the M4 ANE as a 60x increase over the original A11 ANE, and the A18's 35 TOPS as around 58x the A11 figure ^[21][22].

Scroll sideways for more →

Because TOPS is reported at INT8 on most modern Apple SoCs, the chart above is broadly comparable across generations from A14 onward, but the early A11 figure is sometimes quoted in mixed precision and should be read as approximate ^[3]. Apple's per-generation marketing numbers, which describe relative speed-ups on internal benchmarks, line up with what third-party tests have reproduced:

• A14's 16-core ANE runs Apple's image classification benchmark roughly 80% faster than the A12's 8-core part ^[12].
• A15's ANE is about 43% faster than A14 on the same benchmarks ^[14].
• A17 Pro's ANE is roughly 2x A16 ^[19], and the A18 maintains that level ^[22].
• M3's 18 TOPS ANE is up to 60% faster than M1 across a mix of vision and language workloads ^[20], and M4's 38 TOPS ANE is more than 2x faster than M3 ^[21].
• Apple's foundation model on iPhone 15 Pro reports a time-to-first-token of about 0.6 ms per prompt token and a generation rate around 30 tokens per second, running directly on the A17 Pro ANE ^[6].

Energy efficiency is the lever Apple emphasises most. Internal Apple slides for M4 claim equivalent performance to other thin-and-light laptop processors at roughly one-quarter the power draw, and equivalent performance to the M2 at half the power ^[21]. For an iPhone-class workload such as a continuous Visual Look Up scan or live captioning a meeting, ANE execution typically costs single-digit milliwatts per inference, low enough that battery drain is negligible compared with the screen and radios ^[4].

How is the Apple Neural Engine built?

The ANE is a custom Apple design built into the same SoC die as the CPU, GPU, image signal processor, and Secure Enclave. It is fabricated by TSMC on the same leading-edge process node as the rest of the chip: 10 nm for A11, 7 nm for A12 and A13, 5 nm for A14 through A16 and M1 through M3, and 3 nm for A17 Pro, A18, and M4 ^[1][12][19].

While Apple has never published a full instruction set or microarchitecture diagram, third-party reverse engineering, an internal Apple paper on transformer optimisation, and the open apple/ml-ane-transformers repository together describe a fairly consistent picture ^[4][8]:

• The ANE is organised as multiple parallel cores, each containing arrays of multiply-accumulate (MAC) units optimised for the convolutions, matrix multiplications, and elementwise activation functions that dominate neural network inference.
• Cores share a pool of dedicated SRAM that holds activations and weight tiles, with high-bandwidth interfaces to the unified LPDDR memory used by the rest of the SoC.
• Native numerical formats include INT8 and FP16. Newer generations starting around the M3 and M4 add support for additional reduced-precision formats and palletised weight layouts useful for large language models.
• The hardware favours a (B, C, 1, S) channels-first 4D tensor layout. Apple's reference transformer code rewrites linear layers as 2D convolutions and aligns the sequence axis to a 64-byte boundary so that the compiler can map ops directly to the underlying buffers without intermediate transposes ^[8].
• Multi-head attention is split into single-head ops on smaller chunks. This improves L2 residency and lets multiple ANE cores work on different heads in parallel, which is what enables the 10x speed-up Apple reports for transformers on iPhone 13 and later ^[4].
• Power draw under typical inference is on the order of 1 to 2 watts for an iPhone-class ANE and a few watts for the larger M-series ANEs, well below the 5 to 15 watts a discrete GPU or even an integrated GPU would draw to run the same workload ^[3][4].

In practice, the ANE is a feed-forward inference accelerator. It is not designed for training. Backpropagation, gradient computation, and large activation buffers fall back to the GPU or CPU when developers experiment with on-device training, and Apple has so far positioned all the ANE-optimised foundation models as inference-only with adapter layers updated separately ^[6].

How do developers program the Apple Neural Engine?

The public API for the ANE is Core ML, Apple's machine learning framework introduced at WWDC 2017 alongside the A11 Bionic. Developers convert models from PyTorch, TensorFlow, JAX, ONNX, or scikit-learn into the .mlpackage (or older .mlmodel) format using the Python coremltools package ^[5]. At build time, Xcode compiles .mlpackage files into .mlmodelc bundles that are shipped inside an app or downloaded on demand.

When the model is loaded, the Core ML runtime profiles the operation graph, partitions it across the available compute units, and dispatches each subgraph to the ANE, the GPU, or the CPU based on three factors: which ops are supported by the ANE on that device, how the precision and shape constraints are met, and the current power and thermal state. Developers can hint a preferred compute unit (computeUnits = .all, .cpuAndNeuralEngine, .cpuAndGPU, or .cpuOnly) but cannot otherwise force the runtime's hand ^[5]. This is by design: the same model binary can run efficiently across iPhone, iPad, Mac, Vision Pro, and Apple Watch without recompilation.

For transformer workloads, Apple released ane_transformers in 2022, an open reference PyTorch implementation that uses the data layout and chunking tricks described above. On an iPhone 13, a DistilBERT model rewritten with ane_transformers runs about 10x faster and uses about 14x less peak memory than a stock implementation, and lands inference at roughly 3.47 ms per 128-token sequence ^[4][8]. Other research projects, including the Orion framework from Cornell University, characterise the ANE in more detail and explore using it for training as well as inference ^[4].

What is the Neural Engine used for?

The original justification for the ANE was Face ID, a per-unlock face-recognition pipeline that needed to clear the secure enclave, run on-device, and beat the Touch ID latency budget. Once that hardware existed, Apple gradually expanded the workloads it carried.

Scroll sideways for more →

Most of these features run silently. Users do not pick the ANE; the operating system does. The system-wide effect is that on-device ML has become the default, a pattern sometimes called edge AI. Cloud calls are reserved for cases where a model is too large to fit in unified memory or where the user explicitly invokes ChatGPT through Siri.

How does the Apple Neural Engine power Apple Intelligence?

Apple announced Apple Intelligence at WWDC on June 10, 2024 and shipped its first features in iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1 in October 2024 ^[6][7]. The system has two layers:

• An on-device foundation model in the neighbourhood of three billion parameters, fine-tuned with task-specific adapters and quantised aggressively (mixed 2-bit and 4-bit palletised weights, averaging about 3.7 bits per weight), running on the ANE ^[6].
• A larger server-side model running on Apple silicon servers under a system Apple calls Private Cloud Compute (PCC), used for harder requests where the on-device model cannot give a good answer.

Apple Intelligence requires either an A17 Pro or A18 / A18 Pro iPhone, or any Mac, iPad, or Vision Pro with an M-series chip and at least 8 GB of unified memory, plus iOS 18.1 / iPadOS 18.1 / macOS Sequoia 15.1 / visionOS 2.4 or later ^[25]. The 8 GB floor is the binding constraint: the on-device foundation model and its adapter caches need to be paged in alongside the rest of the operating system, which is why the otherwise capable iPhone 15 (A16, 6 GB) is excluded.

The on-device model is not a general chatbot. Apple ships it with adapters for specific tasks: text rewriting and proofreading, summarisation, prioritisation, response suggestions, notification summaries, and Genmoji generation ^[6]. Each adapter is on the order of tens of megabytes and is loaded into the ANE alongside the base model when the relevant feature runs. The full technical detail is laid out in Apple's "Apple Intelligence Foundation Language Models" report and its 2025 update, both published on the Apple Machine Learning Research site ^[6][26].

For reasoning, image generation, and other workloads beyond the on-device model's scope, the request is encrypted, attested, and sent to PCC. Apple publishes the operating system images for those servers and runs them on hardened Apple silicon, so that researchers can verify what is and is not running. The on-device ANE remains the default execution target; PCC only runs when needed and never persists user data after the request is served ^[27].

How does the Apple Neural Engine compare with competing NPUs?

The ANE was one of the first NPUs to ship at scale in a mass-market device. By 2024 nearly every silicon vendor offered a comparable accelerator. The numbers below report each vendor's headline INT8 figure for the most current laptop or smartphone-class chip available in 2024 and early 2025; data-centre accelerators are excluded.

Scroll sideways for more →

The headline TOPS numbers are easy to misread. Vendors report at different precisions and pick the most flattering metric. What sets the ANE apart in practice is less raw throughput and more vertical integration: the same Core ML model runs on every shipping Apple device, the runtime handles fallback automatically, and feature work like Apple Intelligence is wired through the same stack that third-party app developers use ^[3][5]. Qualcomm's Hexagon and Intel's Lunar Lake NPU are competitive on TOPS but require developers to deal with separate SDKs and a less consistent operator coverage story across SoC generations.

What are the limitations of the Apple Neural Engine?

The ANE is a closed, single-vendor accelerator. Some practical consequences:

• Models must go through Core ML conversion. Operations not supported on the current ANE generation fall back to GPU or CPU silently, sometimes turning a fast model into a slow one without an obvious warning ^[5].
• Memory is shared with the rest of the SoC. On iPhones with 6 GB of RAM, large models simply will not fit. This is the reason Apple Intelligence requires 8 GB unified memory and an A17 Pro or later ^[25].
• Developers cannot write CUDA-style kernels for the ANE. Anyone who wants lower-level control has to drop down to the GPU through Metal or use Apple's MLX framework, which targets unified memory rather than the ANE specifically.
• Some PyTorch and transformer ops, especially around dynamic shapes, custom kernels, or non-standard normalisation layers, do not have ANE-friendly implementations and end up on the GPU ^[4].
• The ANE is inference-first. Training large models on-device remains out of scope; even adapter fine-tuning for Apple Intelligence happens on Apple's own infrastructure rather than on user devices ^[6].

None of these are fatal for normal app development, but they explain why projects like the Whisper and Stable Diffusion ports for iOS often spend most of their effort on layout-friendly model rewrites rather than on inference logic.

Why does the Apple Neural Engine matter?

The ANE was one of the first NPUs to ship in a mass-market consumer device, predating most competitors by two to four years and arriving on a billion-plus iPhones over its first half-decade ^[3]. It made on-device AI a default rather than a research demo. Face ID, computational photography, Live Text, and Apple Intelligence all rely on it. It also shifted the privacy conversation around mobile AI: because the inference happens on the user's device, the underlying images, voice samples, or text never need to leave it, and Apple has built a substantial part of its marketing around that fact ^[27].

For Apple itself, the ANE underpins the company's ability to compete in the generative AI cycle without relying on third-party data centres for every interaction. The on-device foundation model in Apple Intelligence runs on the same silicon that handles Face ID and photo Memories, which gives Apple a useful structural advantage: a captive distribution channel of hundreds of millions of devices already shipped with hardware capable of running its models. Whether that translates into a long-term lead over Google's Tensor and Qualcomm's Snapdragon NPUs is still being decided, but as of 2026 the ANE is the most widely deployed NPU in the world, and the most directly responsible for the popularisation of on-device AI ^[3][6].

See also

• Apple Intelligence, Apple silicon, Apple Vision Pro
• Edge AI
• Foundation model, large language model, transformer
• Quantization, edge computing, Gemini Nano
• TensorFlow Lite, Siri, TSMC

References

1. Apple Inc. "The future is here: iPhone X." Apple Newsroom, September 12, 2017. https://www.apple.com/newsroom/2017/09/the-future-is-here-iphone-x/ ↩
2. CNBC. "Apple unveils A11 Bionic neural engine AI chip in iPhone X." September 12, 2017. https://www.cnbc.com/2017/09/12/apple-unveils-a11-bionic-neural-engine-ai-chip-in-iphone-x.html ↩
3. Wikipedia. "Neural Engine." https://en.wikipedia.org/wiki/Neural_Engine ↩
4. Maderix. "Inside the M4 Apple Neural Engine, Part 1: Reverse Engineering." 2024. https://maderix.substack.com/p/inside-the-m4-apple-neural-engine ↩
5. Apple Developer. "Core ML Overview." https://developer.apple.com/documentation/coreml ↩
6. Apple Machine Learning Research. "Introducing Apple's On-Device and Server Foundation Models." June 10, 2024. https://machinelearning.apple.com/research/introducing-apple-foundation-models ↩
7. Apple Inc. "Introducing Apple Intelligence for iPhone, iPad, and Mac." Apple Newsroom, June 10, 2024. https://www.apple.com/newsroom/2024/06/introducing-apple-intelligence-for-iphone-ipad-and-mac/ ↩
8. Apple Machine Learning Research. "Deploying Transformers on the Apple Neural Engine." June 2022. https://machinelearning.apple.com/research/neural-engine-transformers ↩
9. Wikipedia. "Apple A11." https://en.wikipedia.org/wiki/Apple_A11 ↩
10. Wikipedia. "Apple A12." https://en.wikipedia.org/wiki/Apple_A12 ↩
11. Wikipedia. "Apple A13." https://en.wikipedia.org/wiki/Apple_A13 ↩
12. Wikipedia. "Apple A14." https://en.wikipedia.org/wiki/Apple_A14 ↩
13. Apple Inc. "Apple unleashes M1." Apple Newsroom, November 10, 2020. https://www.apple.com/newsroom/2020/11/apple-unleashes-m1/ ↩
14. Wikipedia. "Apple A15." https://en.wikipedia.org/wiki/Apple_A15 ↩
15. Apple Inc. "Apple unveils M1 Pro and M1 Max." Apple Newsroom, October 18, 2021. https://www.apple.com/newsroom/2021/10/introducing-m1-pro-and-m1-max-the-most-powerful-chips-apple-has-ever-built/ ↩
16. Apple Inc. "Apple unveils M2 with breakthrough performance and capabilities." Apple Newsroom, June 6, 2022. https://www.apple.com/newsroom/2022/06/apple-unveils-m2-with-breakthrough-performance-and-capabilities/ ↩
17. Wikipedia. "Apple A16." https://en.wikipedia.org/wiki/Apple_A16 ↩
18. Apple Inc. "Apple introduces M2 Pro and M2 Max." Apple Newsroom, January 17, 2023. https://www.apple.com/newsroom/2023/01/apple-unveils-m2-pro-and-m2-max-next-generation-chips-for-next-level-workflows/ ↩
19. Wikipedia. "Apple A17." https://en.wikipedia.org/wiki/Apple_A17 ↩
20. Apple Inc. "Apple unveils M3, M3 Pro, and M3 Max." Apple Newsroom, October 30, 2023. https://www.apple.com/newsroom/2023/10/apple-unveils-m3-m3-pro-and-m3-max-the-most-advanced-chips-for-a-personal-computer/ ↩
21. Apple Inc. "Apple introduces M4 chip." Apple Newsroom, May 7, 2024. https://www.apple.com/newsroom/2024/05/apple-introduces-m4-chip/ ↩
22. Wikipedia. "Apple A18." https://en.wikipedia.org/wiki/Apple_A18 ↩
23. Apple Inc. "Apple introduces M4 Pro and M4 Max." Apple Newsroom, October 30, 2024. https://www.apple.com/newsroom/2024/10/new-macbook-pro-features-m4-family-of-chips-and-apple-intelligence/ ↩
24. Apple Inc. "Apple unveils new Mac Studio, the most powerful Mac ever." Apple Newsroom, March 5, 2025. https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac-studio-the-most-powerful-mac-ever/ ↩
25. Apple Support. "How to get Apple Intelligence." https://support.apple.com/en-us/121115 ↩
26. Apple Machine Learning Research. "Apple Intelligence Foundation Language Models Tech Report 2025." https://machinelearning.apple.com/research/apple-foundation-models-tech-report-2025 ↩
27. Apple Inc. "Private Cloud Compute: A new frontier for AI privacy in the cloud." Apple Security Research Blog, June 10, 2024. https://security.apple.com/blog/private-cloud-compute/ ↩
28. Tom's Hardware. "Qualcomm details its Snapdragon X Elite: all SKUs have NPU with 45 TOPS for AI workloads." 2024. https://www.tomshardware.com/pc-components/cpus/qualcomm-snapdragon-x-elite-npu-45-tops ↩
29. Google. "Tensor G4 and the Pixel 9 Tensor Processing Unit." Google Blog, August 13, 2024. https://blog.google/products/pixel/google-pixel-9-tensor-g4-features/ ↩
30. The Register. "Apple unveils M4 chip with neural engine capable of 38 TOPS." May 7, 2024. https://www.theregister.com/2024/05/07/apple_m4_ipad/ ↩