MLX

MLX is an open-source array and machine-learning framework built by Apple Inc.'s machine-learning research team specifically for Apple silicon, the M1, M2, M3, M4, and M5 series of system-on-chip processors. First released on GitHub on December 5, 2023 under the permissive MIT license, MLX pairs a NumPy-like Python API for arrays with a PyTorch-like API for neural networks, and it exploits the unified-memory architecture of Apple silicon so that arrays are shared between CPU and GPU without explicit data copies.^[1][20] The framework's official description is that "MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research."^[1] As of November 2026 the repository had about 29,000 stars on GitHub, and the latest release was version 0.32.4 (October 14, 2026).^[1][26]

MLX was conceived as a research-friendly framework that closes the gap between CPU and GPU programming on the Mac. On most other platforms, an ML developer has to think about which device a tensor lives on and explicitly move it across an interconnect such as PCIe. On Apple silicon, CPU and GPU share the same physical memory, so the framework can avoid those copies entirely. MLX bakes that assumption into its core, which is what makes it noticeably different from cross-platform frameworks like PyTorch with the Metal Performance Shaders backend or JAX with its experimental Metal backend.^[1]

Since the initial release, MLX has grown into a small ecosystem of related projects: the core mlx array library, the mlx-lm package for large language model inference and fine-tuning, the mlx-data library for data loading, the mlx-swift package that exposes the API to Swift and iOS, and the mlx-examples repository that hosts reference implementations of models such as Llama, Mistral, Mixtral, Stable Diffusion, Whisper, and many others.^[2][3] By 2026 the MLX Community organization on Hugging Face hosted thousands of pre-converted model weights.^[11]

When was MLX released?

MLX appeared during a period when running large language models locally on a Mac was becoming popular but was mostly limited to projects like llama.cpp that targeted the CPU plus a thin Metal layer. Apple's ML research division wanted a framework that researchers inside and outside the company could actually train and prototype with on consumer hardware, not just deploy to. The result was MLX, released on December 5, 2023 alongside the mlx-examples repository so that the framework would launch with working reference code rather than as a bare runtime.^[2][20]

Who created MLX?

MLX was created by a small team inside Apple's machine-learning research division. The four engineers credited with equal contribution on the initial release are Awni Hannun, Jagrit Digani, Angelos Katharopoulos, and Ronan Collobert.^[1]

Awni Hannun acted as the public face of the project for its first two years. Before joining Apple, he worked at Facebook AI Research and earned his PhD at Stanford University, where he co-authored the Deep Speech speech-recognition papers in Andrew Ng's group. He announced the original release on X on December 5, 2023, writing that "MLX is an efficient machine learning framework specifically designed for Apple silicon (i.e. your laptop!)" and continued posting roadmap updates, demos, and benchmarks for the next several years.^[7] Hannun left Apple in February 2026 but, in his farewell post on February 27, 2026, named the people continuing to maintain MLX (Angelos Katharopoulos, Cheng Zhao, Jagrit Digani, and others) and said the project would keep moving.^[17]

Ronan Collobert is a longtime researcher in deep learning who previously worked at Facebook AI Research and on the Torch and Flashlight frameworks. Angelos Katharopoulos is known for work on linear-attention transformers and contributes heavily to MLX's compiler and distributed back-ends. Jagrit Digani has focused on Metal kernels and operator coverage. Outside contributions come from a wide community of Mac-using ML developers, with the GitHub project showing thousands of merged pull requests across its first three years.^[1]

How does MLX work? Architecture and design philosophy

MLX is built around a small set of design decisions that, taken together, distinguish it from PyTorch, JAX, and TensorFlow. The most consequential is unified memory: there is no .to("cuda") and no separate host and device tensor, because on Apple silicon the CPU, GPU, and Neural Engine all address the same physical RAM. Operations specify the device they should run on, but the data does not move.^[1]

The official documentation summarizes the model in one line: "Computations in MLX are lazy. Arrays are only materialized when needed."^[4] Lazy evaluation matters because it lets MLX fuse operators and skip work whose results are never read. It also makes function transformations cleaner, since the framework already has a graph to differentiate, vectorize, or compile, and MLX allows arbitrary compositions such as grad(vmap(grad(fn))).^[1][4] The trade-off is that the user has to remember to call mx.eval (or perform any operation that materializes a value) when they want a side effect like printing or timing.

The Python package mlx.nn defines familiar building blocks: Linear, Embedding, Conv1d, Conv2d, Conv3d, LSTM, GRU, Transformer, MultiHeadAttention, RMSNorm, LayerNorm, RoPE, and the usual activation functions. The mlx.optimizers package provides SGD, Adam, AdamW, Lion, Muon, and Adafactor, plus learning-rate schedules such as cosine decay, exponential decay, and linear schedules. Anyone who has used PyTorch can read MLX neural-network code without much trouble.^[1]

What are MLX's key features?

The headline features that come up most often in tutorials and benchmarks are listed below.

One practical detail that surprises new users is that MLX does not use the Apple Neural Engine (Apple Neural Engine, or ANE) directly. The ANE is exposed through Core ML, and MLX runs on the CPU and GPU instead. From the M5 onward, however, the GPU itself contains neural accelerators that provide matrix-multiplication primitives, and MLX uses those through Metal 4's Tensor Operations.^[6]

What is mlx-lm used for?

mlx-lm is the package most users actually install. It is a high-level Python library and command-line interface for running, quantizing, and fine-tuning large language models on Apple silicon. The default model for generation and chat is a 4-bit quantized Llama-3.2-3B-Instruct, so a user can type pip install mlx-lm followed by mlx_lm.chat and get a working chatbot in a few minutes on any modern Mac.^[3]

The package supports thousands of open-weight models on the Hugging Face Hub. The most commonly used families include Llama 1, Llama 2, Llama 3, Llama 3.1, Llama 3.2, Llama 4, Mistral 7B, Mixtral 8x7B and 8x22B, Qwen 1.5, Qwen2, Qwen2.5, Qwen3, Phi-2 and Phi-3, Gemma and Gemma 2, DeepSeek (including DeepSeek R1 and V3 in distributed mode), Yi, Stable LM, and Falcon. Vision-language models are covered by the related community package mlx-vlm.^[3][11]

The mlx-lm CLI exposes the main workflows directly:

• mlx_lm.generate for one-shot text generation with a prompt
• mlx_lm.chat for an interactive REPL-style chat session
• mlx_lm.server for a local OpenAI-compatible HTTP server
• mlx_lm.convert for downloading a Hugging Face model and quantizing it to 2-bit, 3-bit, 4-bit, 6-bit, or 8-bit
• mlx_lm.lora for low-rank adapter fine-tuning, including QLoRA on top of a quantized base model
• mlx_lm.fuse for merging a trained LoRA adapter back into the base weights
• mlx_lm.upload for pushing a converted model to the Hugging Face Hub

Under the hood, mlx-lm uses streaming generation with a key-value cache, supports prompt caching to make repeated prefixes free, and offers a rotating KV cache to keep memory bounded for very long contexts. Distributed inference is available via mx.distributed, which is how Awni Hannun published instructions in January 2025 for running DeepSeek V3 and R1 across a cluster of Macs, and how Apple ran the 816-billion-parameter Muse Spark Open across four Mac Studios in October 2026.^[16][27]

The mlx-data subproject

mlx-data is a separate library for building data-loading and preprocessing pipelines. It is framework-agnostic, meaning that although it ships from the ml-explore organization it can be used with PyTorch or JAX in addition to MLX. The library focuses on streaming, sharding, and on-the-fly transformations such as resizing images, tokenizing text, and decoding audio. It is written in C++ with Python bindings and is designed to keep up with the throughput that Apple silicon GPUs can reach, especially for image and audio workloads.

The mlx-examples repository

mlx-examples is a separate Apple-maintained GitHub repository that ships reference implementations rather than reusable APIs. It is the place most people go first to see how a model is meant to be expressed in MLX.^[2]

Notable examples include:

These examples are a major reason MLX got traction quickly: a developer who wanted to run Stable Diffusion or Llama on a MacBook could clone the repo and have a working pipeline within an hour of the framework's release.^[2]

How fast is MLX?

MLX is fast on Apple silicon, particularly for inference. The exact numbers depend on chip, model, batch size, and quantization, but a few patterns show up consistently across community benchmarks.

Apple's own benchmarks give a sense of scale. On a 24 GB M5 MacBook Pro with a 4,096-token prompt, MLX keeps time-to-first-token under 10 seconds for a 4-bit 14B dense language model and under 3 seconds for a 30B mixture-of-experts model,^[6] and on a Mac Studio with M5 Ultra, decode throughput in tokens per second is about 1.6 times that of an M3 Ultra.^[27] On the M5, Apple's own research blog reports that the GPU's neural accelerators yield "up to 4x speedup compared to a M4 baseline for time-to-first-token in language model inference," with subsequent token generation 19 to 27 percent faster, driven mostly by the M5's higher memory bandwidth (153 GB/s versus 120 GB/s, about 28 percent more) and the new Metal 4 Tensor Operations.^[6] Apple explains the split this way: prompt processing "takes full advantage of the Neural Accelerators," while "generating subsequent tokens is bounded by memory bandwidth, rather than by compute ability."^[6]

In head-to-head comparisons, MLX tends to beat PyTorch with the MPS backend for inference and many single-array operations, while PyTorch can still be faster for some training workloads. One frequently cited Towards Data Science benchmark found PyTorch MPS at roughly 10 to 14 seconds per training epoch for a small CNN, with the MLX version of the same model at roughly 21 to 27 seconds, so the framework has not entirely closed the gap on every workload.^[12] For inference and serving, MLX is generally the fastest option among Apple silicon native frameworks. The clearest signal came on March 30, 2026, when Ollama 0.19 previewed an MLX-powered engine for Apple silicon: in Ollama's test of Qwen3.5-35B-A3B, decode speed rose from 58 tokens per second (Ollama 0.18, Q4_K_M) to 112 tokens per second (Ollama 0.19, NVFP4), and the preview asks for a Mac with more than 32 GB of unified memory.^[18]

Nvidia GPUs are still faster in absolute terms on many workloads; Tristan Bilot's operation-level benchmark compares MLX on ten Apple silicon chips with PyTorch on three CUDA GPUs.^[13] Apple's case rests partly on power: in its October 2026 tests, a Mac Studio peaked at about 200 W (M3 Ultra) or 300 W (M5 Ultra) during LLM inference.^[27]

How does MLX differ from PyTorch and llama.cpp?

MLX sits in a fairly crowded landscape, but its niche of "first-class Apple silicon framework that looks like NumPy and PyTorch" is well defined. The table below compares it with the frameworks it is most often weighed against.

A quick summary of how to think about each:

• MLX is the choice when you want to write training or inference code that looks like PyTorch but runs natively on Apple silicon, especially if you want to use unified memory and fine-tune on the same machine you developed on.
• PyTorch with MPS is the choice when you want code that also runs on Linux servers with CUDA. The MPS back-end is mature enough to be the default "PyTorch on Mac" experience.
• Core ML is the choice when you are shipping a model inside an iOS or macOS app. It can use the Apple Neural Engine, which MLX cannot.
• llama.cpp is the choice when you want maximum portability for LLM inference, especially across non-Apple hardware. Its GGUF file format has become a de facto standard, and as of Ollama 0.19 it covers far more model architectures than the newer MLX runner.^[18]
• The Apple Foundation Models framework is the choice when you want to call Apple's own on-device LLM (the one inside Apple Intelligence) without managing weights or quantization yourself.

What is MLX used for?

MLX has settled into a few clearly identified use cases over its first years:

• Running open-weight LLMs locally on a Mac, often as a faster alternative to llama.cpp or Ollama (Ollama itself now supports MLX as a back-end)^[18]
• Local fine-tuning, especially with LoRA and QLoRA, so a developer can adapt a 7B or 13B model to their own data overnight on a MacBook Pro
• Research prototyping on Apple silicon, where the unified-memory model lets researchers handle larger batches than the same wattage of a discrete GPU would allow
• Distributed inference and training across multiple Mac Studios, particularly for very large MoE models that do not fit on a single machine^[15][16]
• Embedded use in Mac and iOS apps via the Swift API, for example for on-device transcription, image generation, or chat^[8]
• Teaching and learning, since the API is simple and the lazy-graph model makes it relatively easy to inspect what a transformation is doing

Strengths

The main reasons people pick MLX over alternatives:

• Native to Apple silicon, so kernels are written for the actual hardware rather than translated through a portable abstraction layer
• Unified memory removes a class of mistakes around device placement and copies
• API is genuinely familiar: a PyTorch user can read MLX code on the first try, and a NumPy user can write array operations with no extra learning
• MIT license, which means companies can use it without legal friction
• Active maintenance from a small but committed Apple team, with frequent releases
• Excellent inference performance for quantized LLMs on consumer Macs
• Tight integration with Hugging Face through the mlx-community organization, which had thousands of pre-converted models by 2026^[10][11]

Weaknesses and limitations

The usual caveats:

• Apple silicon only for the primary back-end; the CUDA back-end exists but is newer and not the focus of development
• Smaller community than PyTorch or JAX, so there are fewer tutorials, fewer Stack Overflow answers, and fewer pretrained MLX-native checkpoints (though Hugging Face conversions help)
• Distributed training is younger than the equivalent in PyTorch (FSDP, DeepSpeed) and is still evolving
• Not the choice for production cloud serving, where vLLM, TensorRT-LLM, and PyTorch dominate
• Does not target the Apple Neural Engine; for that you still need Core ML
• Some training workloads remain faster on PyTorch with MPS, so users sometimes write training in PyTorch and inference in MLX^[12]

How widely is MLX adopted?

MLX has been adopted broadly within the Mac-using ML community. The GitHub repository had about 29,000 stars and 2,400 forks as of November 2026; the mlx-community Hugging Face organization had thousands of converted model checkpoints with thousands of members, and major tools like Ollama added MLX as an optional back-end.^[1][11][18] Apple's WWDC 2024 and 2025 sessions present MLX to developers as a way to train, fine-tune and run models on Mac; the 2025 session is devoted to running and fine-tuning large language models with MLX LM.^[9][29]

Third-party libraries extend MLX beyond text. Prince Canuma published mlx-vlm, for running and fine-tuning vision-language models, in April 2024, and mlx-audio, for text-to-speech, speech-to-text and speech-to-speech, had its first PyPI release in February 2025; LM Studio uses mlx-vlm for vision models in its MLX engine.^[21][22][23] On October 27, 2026, Canuma announced Neywa Labs, a startup that builds software for running AI models locally on Apple silicon and lists both libraries among its open-source engines.^[24][25]

Outside Apple, MLX shows up in academic work on private LLMs (for example, multi-node mixture-of-experts research using Apple silicon clusters) and in many indie developer tools that ship on-device AI features. Multi-Mac demonstrations of very large models, from DeepSeek R1 on three M2 Ultra Mac Studios in January 2025 to Apple's October 2026 run of the 816-billion-parameter Muse Spark Open across four M5 Ultra Mac Studios, have pushed Apple's small-cluster story as a low-cost alternative to Nvidia DGX boxes for some workloads.^[16][27]

In February 2026, Awni Hannun, the public face of MLX, announced that he was leaving Apple, writing "Today is my last day at Apple. Building MLX with our amazing team and community has been an absolute pleasure. It's still early days for AI on Apple silicon."^[17] Coverage at the time placed his exit after at least a dozen AI researcher departures from Apple since early 2025.^[28] Hannun named the engineers continuing to maintain MLX, and the project has continued to ship releases since.^[17]

Where MLX fits in Apple's ML stack

Apple has several machine-learning frameworks, and they serve different jobs.

In practice, a developer might prototype a model in MLX, convert it to Core ML for shipping in a consumer app, and call Apple's Foundation Models framework when they just need general-purpose text from the system LLM rather than their own custom model.

Is MLX open source? License and governance

MLX is released under the MIT license, which permits commercial use, modification, distribution, and private use without requiring derivative works to be open-sourced.^[1][5] The project is maintained on GitHub under the ml-explore organization, which is owned by Apple. Releases are cut directly from the main branch, and contributions follow a standard pull-request workflow. The mlx-examples, mlx-lm, mlx-data, mlx-swift, and mlx-c repositories use the same MIT license and the same governance model.^[5]

Recent context (2024 to 2026)

By 2026 MLX has settled in as the standard Mac-native ML framework for local LLM development and on-device fine-tuning. The Hugging Face community treats it as a first-class target, alongside the PyTorch and GGUF formats.^[10] Apple continues to invest in the project: distributed training and inference across multiple Macs has matured, quantization libraries have grown to include 2-bit and 3-bit variants, the Swift bindings are good enough to ship inside iOS and macOS apps, and Metal 4's Tensor Operations let MLX exploit the new neural accelerators in the M5 GPU.^[6] The CUDA back-end added in 2025 also widened MLX's appeal: the same code can prototype on a MacBook and then run on Nvidia hardware in a cloud environment.

The departure of Awni Hannun in early 2026 left an open question about Apple's long-term commitment to MLX, but the project has continued to ship releases (the 0.31 and 0.32 series through October 2026), with work from maintainers Hannun named such as Angelos Katharopoulos and Cheng Zhao.^[17][26] For most practical purposes, if you want to run or train a model on a Mac in 2026, MLX is the framework people reach for first.

Continue reading

• PyTorchPyTorch is an open-source software library for tensor computation and machine learning.Mentioned 29 times above
• AApple SiliconApple silicon is the family of ARM architecture system on a chip (SoC) and system in a package (SiP) processors that Apple Inc. designs for its own productsMentioned 25 times above
• OllamaOllama is a free, open-source runtime for downloading, running, and managing open-weight large language models (LLMs) locally on personal computers and servers.Mentioned 8 times above
• JAXJAX is an open-source Python library for accelerator-oriented array computation and program transformation.Mentioned 7 times above
• CCore MLCore ML is Apple's framework for running trained machine learning models on-device across iOS, iPadOS, macOS, watchOS, tvOS, and visionOS.Mentioned 6 times above
• AApple Neural EngineThe Apple Neural Engine (ANE) is the dedicated neural network accelerator, a type of neural processing unit (NPU), that Apple builds into the Apple silicon system-on-chip powering iPhone, iPad, Mac, Apple…Mentioned 5 times above

More to read

• llama.cpp
• Apple Intelligence
• NumPy
• Apple Inc.
• tinygrad
• PyTorch Lightning

Topics:AI CompaniesAI HardwareDeveloper ToolsOpen Source AI

See also

• Apple Inc.
• Apple silicon
• Apple Intelligence
• Apple Neural Engine
• PyTorch
• PyTorch Lightning
• JAX
• NumPy
• Core ML
• llama.cpp
• Ollama
• tinygrad
• Ray (framework)
• Haystack (framework)
• Local LLM VRAM and RAM calculator

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12ml-explore. "MLX: An array framework for Apple silicon." GitHub repository. github.com/...mlx
2. ^1 ^2 ^3 ^4ml-explore. "MLX Examples." GitHub repository. github.com/...mlx-examples
3. ^1 ^2 ^3ml-explore. "MLX-LM: Run LLMs with MLX." GitHub repository. github.com/...mlx-lm
4. ^1 ^2ml-explore. "MLX Documentation." ml-explore.github.io/...html
5. ^1 ^2Apple Open Source. "MLX project page." opensource.apple.com/...mlx
6. ^1 ^2 ^3 ^4 ^5 ^6Apple Machine Learning Research. "Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU." machinelearning.apple.com/...exploring-llms-mlx-m5
7. ^1 ^2Awni Hannun. "MLX is an efficient machine learning framework specifically designed for Apple silicon." X (Twitter), December 5, 2023. x.com/...1732184443451019431
8. ^1 ^2Swift.org. "On-device ML research with MLX and Swift." Blog post, February 20, 2024. swift.org/...mlx-swift
9. ^Apple Developer. "Explore large language models on Apple silicon with MLX." WWDC 2025 session 298. developer.apple.com/...298
10. ^1 ^2Hugging Face. "Using MLX at Hugging Face." Hub documentation. huggingface.co/...mlx
11. ^1 ^2 ^3 ^4 ^5Hugging Face. "MLX Community organization." huggingface.co/mlx-community
12. ^1 ^2Mike Cvet. "PyTorch and MLX for Apple Silicon." Towards Data Science. towardsdatascience.com/...ple-silicon-4f35b9f60e39
13. ^Tristan Bilot. "How Fast Is MLX? A Comprehensive Benchmark on Apple Silicon Chips and CUDA GPUs." Towards Data Science. medium.com/...n-chips-and-4-cuda-gpus-378a0ae356a0
14. Apple Machine Learning Research. "Introducing Apple's On-Device and Server Foundation Models." machinelearning.apple.com/...ple-foundation-models
15. ^1 ^2Awni Hannun. "The latest MLX is out! And it has a new distributed back-end (JACCL) that uses RDMA over TB5." X (Twitter), December 18, 2025. x.com/...2001667839539978580
16. ^1 ^2 ^3 ^4Simon Willison. "Run DeepSeek R1 or V3 with MLX Distributed." Blog post, January 22, 2025. simonwillison.net/...mlx-distributed
17. ^1 ^2 ^3 ^4Awni Hannun. "Today is my last day at Apple." X (Twitter), February 27, 2026. x.com/...2027506228143030480
18. ^1 ^2 ^3 ^4Ollama. "Ollama is now powered by MLX on Apple Silicon in preview." Blog post, March 2026. ollama.com/...mlx
19. Apple Inc. "Apple introduces Apple Intelligence." WWDC 2024 announcement coverage.
20. ^1 ^2 ^3Computerworld. "Apple launches MLX machine-learning framework for Apple Silicon." December 2023. computerworld.com/...g-framework-for-apple-silicon
21. ^Blaizzy/mlx-vlm, GitHub repository and README. Accessed October 29, 2026. github.com/...mlx-vlm
22. ^Blaizzy/mlx-audio, GitHub repository and README. Accessed October 29, 2026. github.com/...mlx-audio
23. ^lmstudio-ai/mlx-engine, GitHub README. Accessed October 29, 2026. github.com/...mlx-engine
24. ^Prince Canuma, "Today we're coming out of stealth, introducing Neywa Labs," X, October 27, 2026. x.com/...2115128700379566155
25. ^Neywa Labs, "About Neywa Labs: Making Frontier AI Native" (company page). Accessed October 29, 2026. neywalabs.ai/company
26. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11ml-explore. "Releases." GitHub, ml-explore/mlx. Accessed November 5, 2026. github.com/...releases
27. ^1 ^2 ^3 ^4 ^5Apple Machine Learning Research. "Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio." October 15, 2026. machinelearning.apple.com/...distributed-inference
28. ^Awesome Agents. "The Creator of MLX Just Left Apple - And He's Not the First." February 2026. awesomeagents.ai/...annun-mlx-creator-leaves-apple
29. ^1 ^2Apple Developer. "Explore machine learning on Apple platforms." WWDC 2024 session 10223. developer.apple.com/...10223
30. ^Apple Machine Learning Research. "Updates to Apple's On-Device and Server Foundation Language Models." June 9, 2025. machinelearning.apple.com/...n-models-2025-updates

Show all 30 references