MindSpore

MindSpore is an open-source deep learning framework developed by Huawei that Huawei describes as "a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios" ^[3]. Huawei open-sourced it on 28 March 2020 at the Huawei Developer Conference (HDC.Cloud), positioning it as an "all-scenario" framework that runs across device, edge, and cloud with a single programming model and as a Chinese alternative to Google's TensorFlow and Meta's PyTorch ^[1][2][12]. MindSpore provides first-class, co-optimized support for Huawei's Ascend AI processors and also runs on CPU and GPU backends ^[2][3]. Its design centers on three capabilities Huawei emphasized from the start: source-to-source automatic differentiation, automatic parallelism for large-scale distributed training, and a unified execution model that mixes dynamic and static computational graphs ^[3][4]. The framework is released under the Apache License 2.0 and is the framework used to train Huawei's PanGu large language models ^[3][5].

What is MindSpore?

MindSpore is Huawei's flagship open-source deep learning framework, the software counterpart to its Ascend AI hardware. It is a full training and inference stack that lets developers define neural networks in Python and run them across the full "device, edge, and cloud" range of deployment targets from a single codebase ^[2][3]. Huawei positions MindSpore directly against the two dominant Western frameworks: at launch it was framed as a rival to TensorFlow and PyTorch, and Chinese-ecosystem assessments have called it "the most active, most paid attention to, and most used framework in the Chinese community" ^[2][12]. In practical terms, MindSpore is the framework most likely to be used by teams building on Huawei's Ascend accelerators, for which it is the natively supported and most heavily optimized option.

Huawei has highlighted code-efficiency gains as a selling point. The company has stated that for a typical natural-language-processing network MindSpore uses about 20% fewer lines of core code than leading frameworks while improving development efficiency by more than 50% ^[2][13].

When was MindSpore released and open-sourced?

Huawei first showed MindSpore publicly in August 2019, when it presented the framework alongside the Ascend 910 training chip and the Ascend 310 inference chip as part of its full-stack AI strategy ^[2][12]. The open-source release followed on 28 March 2020 at HDC.Cloud. The announcement was made by Professor Chen Lei, a chief scientist on the MindSpore team, who described it as China's first all-scenario AI computing framework ^[1][2]. The code was published on the Chinese hosting platform Gitee, with a mirror on GitHub, and MindSpore went on to become the top open-source project on Gitee in 2020 ^[1][2]. Huawei stated that MindSpore had been developed together with external collaborators including Imperial College London, Peking University, and the University of Edinburgh ^[1].

The same launch introduced two companion tools that ship alongside the core framework: MindInsight, a debugging and visualization tool for inspecting the training process, and MindArmour, a module for adversarial robustness and model security testing ^[1].

The first publicly available package was version 0.1.0, uploaded on 28 March 2020 ^[6]. The project moved quickly through its 0.x and 1.x series before reaching a 2.0 milestone in 2023, after which development continued on a roughly biannual cadence.

Scroll sideways for more →

How is MindSpore designed and structured?

MindSpore is written mainly in C++ and Python, with additional language components, and exposes a Python API for building models ^[4]. Its architecture is usually described in layers. At the top is a model and suite layer that bundles ready-made models and development kits such as MindSpore Transformers and scientific-computing libraries. Below that is the Python expression layer (the user-facing API for defining networks, control flow, and custom operators in native Python). Underneath sits the framework core: tensor data structures, operators, the automatic-differentiation engine, the parallel-computing modules, and the compiler and runtime ^[4].

A central element is MindIR, MindSpore's intermediate representation. User code is parsed into this functional IR, which the compiler (sometimes referred to as MindCompiler) then optimizes. The IR is the substrate for hardware-independent passes such as automatic differentiation and for hardware-related passes such as automatic parallelism and operator fusion ^[3][4]. Because MindIR is a functional representation, it has a direct correspondence to the composition of mathematical functions, which is what makes the source-to-source differentiation approach tractable ^[3].

MindSpore supports two execution modes. PyNative mode runs operations eagerly, op by op, which is convenient for debugging and matches the imperative style familiar from other frameworks. Graph mode (GRAPH_MODE) compiles the network into a static graph for whole-graph optimization and better performance, particularly on Ascend hardware. The framework converts Python source into graphs via abstract-syntax-tree analysis, so the same code can run in either mode and developers can switch between them ^[3][4].

How do automatic differentiation and automatic parallelism work in MindSpore?

MindSpore's automatic differentiation uses source transformation rather than operator overloading. The framework parses the Python function or Cell object to be differentiated, builds a differentiable function object in MindIR, and derives gradients by following the call chain ^[3]. Huawei describes the approach this way: "The implementation of MindSpore automatic differentiation can be understood as the symbolic differentiation of the program itself" ^[4]. Because MindIR is a functional intermediate expression with a direct correspondence to composite functions in basic algebra, this lets MindSpore differentiate through native Python control flow such as if, while, and for constructs while still allowing static compilation and optimization ^[3][4].

Automatic parallelism is one of MindSpore's most distinctive features. The aim is to let a user write what looks like serial, single-device code and have the framework distribute it across many accelerators ^[2][3]. MindSpore supports several parallel strategies that can be combined: data parallelism, operator-level model parallelism, pipeline parallelism, and optimizer parallelism ^[4]. It uses a fine-grained approach that can split individual operators across devices. A key mechanism here is tensor redistribution, which automatically inserts the operations needed to convert the device layout of a tensor produced by one operator into the layout expected by the next, so that mixed strategies can be expressed uniformly ^[4]. In its automatic mode, MindSpore can search for a low-cost partitioning strategy rather than requiring the user to specify the sharding by hand, so as to "automatically select a least cost model splitting strategy" for distributed training ^[3][4].

How does MindSpore relate to Ascend?

MindSpore is the software half of Huawei's AI stack and is co-designed with the Ascend processors and the CANN (Compute Architecture for Neural Networks) software layer that sits between the framework and the hardware ^[3]. Huawei has emphasized native support for Ascend, with the graph engine compiling MindSpore graphs to run efficiently on the chips ^[2]. Because MindSpore also targets CPU and GPU, models can be developed on commodity hardware and then deployed to Ascend, but the framework is most tightly optimized for the Ascend line, including the Ascend 910 training accelerator ^[3][5].

The table below summarizes the hardware backends MindSpore targets.

Scroll sideways for more →

What is in the MindSpore ecosystem?

Beyond the core training and inference engine, MindSpore has grown a set of suites and tools. The best known is MindSpore Transformers, also called MindFormers, a full-process development suite for pretraining, fine-tuning, evaluating, and deploying large Transformer models ^[9]. MindFormers builds on the framework's parallel capabilities and supports combinations of data, model, optimizer, pipeline, sequence, context, and mixture-of-experts (MoE) parallelism, and it interoperates with mainstream ecosystems such as Hugging Face, Megatron-LM, vLLM, and OpenCompass ^[9].

For deployment outside the data center, MindSpore Lite provides a lightweight runtime for inference and training on mobile, embedded, and edge devices, which is the part of the framework that delivers on the "device" side of the all-scenario design ^[3]. The launch-day companions MindInsight (debugging and visualization) and MindArmour (model security and adversarial robustness) remain part of the wider project ^[1]. There are also domain-specific science kits in the MindSpore family, such as MindFlow for computational fluid dynamics and MindElec for electromagnetic simulation, reflecting Huawei's interest in AI for scientific computing ^[8].

What makes MindSpore different from PyTorch and TensorFlow?

MindSpore occupies similar ground to PyTorch and TensorFlow, but its distinguishing choices follow from Huawei's hardware strategy. Where TensorFlow 1.x was built around static graphs and PyTorch around eager (dynamic) execution, MindSpore tries to unify both modes behind one Python API so that the same model definition can run eagerly for debugging or be compiled to a static graph for performance ^[2][4]. Its source-to-source automatic differentiation contrasts with the tape-based, operator-overloading autograd used by PyTorch, and it is intended to differentiate through Python control flow while preserving opportunities for static optimization ^[3]. The automatic-parallelism system, which can shard a model across many devices from serial-looking code, is positioned as an answer to the manual sharding work that large-model training otherwise requires ^[3]. Huawei also points to code economy: the company reports that for a representative NLP network MindSpore needs roughly 20% fewer lines of core code (about 2,000 lines versus about 2,500 in TensorFlow) while raising development efficiency by more than 50% ^[2][13]. In practice MindSpore's clearest advantage is on Huawei's own Ascend processors, for which it is the natively supported and most heavily optimized framework ^[2][3].

Scroll sideways for more →

Who uses MindSpore?

MindSpore is the training framework behind Huawei's PanGu family of large models and several other Chinese research models. PanGu-Alpha, a Chinese autoregressive language model described in a 2021 paper, was trained on a cluster of 2,048 Ascend 910 processors using MindSpore's automatic-parallelism system, which the authors combined across five parallelism dimensions ^[5]. The later PanGu-Sigma, a sparse model with 1.085 trillion parameters, was implemented with MindSpore (version 1.6) and trained on 512 Ascend 910 accelerators over more than 100 days, processing about 329 billion tokens ^[10]. CodeGeeX, a multilingual code-generation model from Tsinghua University and Zhipu AI, was trained on a cluster of 1,536 Ascend 910 processors using MindSpore version 1.7, consuming roughly 850 billion tokens over about two months ^[7].

Huawei has reported that the MindSpore open-source community grew to more than 170,000 developers with over 240,000 downloads, used across more than ten industries, though such figures come from Huawei itself and should be read as the company's own community numbers rather than independently audited usage ^[11].

Is MindSpore open source?

Yes. MindSpore is released under the Apache License 2.0, a permissive open-source license that allows use, modification, and redistribution, including in commercial and proprietary products, subject to attribution and the license's patent and notice terms ^[3][4]. The source is hosted on Gitee (gitee.com/mindspore/mindspore) with a mirror on GitHub, and releases are distributed through PyPI as well ^[4][6].

References

1. "MindSpore Goes Open Source, Empowering Global Developers with an All-Scenario AI Computing Framework." WebWire, 28 March 2020. https://www.webwire.com/ViewPressRel.asp?aId=257293 ↩
2. "MindSpore: Huawei's All-Scenario AI Computing Framework Now Open Source." CIO / IDG. https://www.cio.com/article/193320/mindspore-huaweis-all-scenario-ai-computing-framework-now-open-source.html ↩
3. "MindSpore" (GitHub repository README). mindspore-ai/mindspore, GitHub. https://github.com/mindspore-ai/mindspore ↩
4. "MindSpore" entry. Wikipedia. https://en.wikipedia.org/wiki/MindSpore ↩
5. Wei Zeng et al. "PanGu-Alpha: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation." arXiv:2104.12369. https://arxiv.org/abs/2104.12369 ↩
6. "mindspore" release history. Python Package Index (PyPI). https://pypi.org/project/mindspore/#history ↩
7. Qinkai Zheng et al. "CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X." arXiv:2303.17568. https://arxiv.org/abs/2303.17568 ↩
8. "MindSpore 2.0.0 Release Notes." MindSpore documentation. https://www.mindspore.cn/docs/en/r2.0/RELEASE.html ↩
9. "MindSpore Transformers (MindFormers)." mindspore-lab/mindformers, GitHub. https://github.com/mindspore-lab/mindformers ↩
10. Xiaozhe Ren et al. "PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing." arXiv:2303.10845. https://arxiv.org/abs/2303.10845 ↩
11. "Huawei MindSpore (the platform for development of AI)." TAdviser. https://tadviser.com/index.php/Product:Huawei_MindSpore_(the_platform_for_development_of_AI) ↩
12. "Huawei open-sources AI framework MindSpore to rival Google's TensorFlow." SiliconANGLE, 30 March 2020. https://siliconangle.com/2020/03/30/huawei-open-sourced-ai-framework-called-mindspore-rival-googles-tensorflow/ ↩
13. "Huawei open sources MindSpore: claims to provide 'all-scenario AI computing framework'." TelecomTV. https://www.telecomtv.com/content/ai-ml/huawei-open-sources-mindspore-claims-to-provide-all-scenario-ai-computing-framework-38188/ ↩