# Qwen2.5-Math

> Source: https://aiwiki.ai/wiki/qwen2_5_math
> Updated: 2026-07-24
> Fact-checked: 2026-07-24
> Categories: Chinese AI, Large Language Models, Mathematics, Open Source AI
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Qwen2.5-Math." aiwiki.ai, 24 Jul 2026. https://aiwiki.ai/wiki/qwen2_5_math
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

Qwen2.5-Math is a family of mathematics-specialized [large language models](https://aiwiki.ai/wiki/large_language_model) developed by the Qwen team at [Alibaba Cloud](https://aiwiki.ai/wiki/alibaba_cloud) and released in September 2024. The series consists of base models at 1.5B, 7B, and 72B parameters, an instruction-tuned version of each, and a 72B mathematical [reward model](https://aiwiki.ai/wiki/reward_model), Qwen2.5-Math-RM-72B.[1][2] Unlike its predecessor [Qwen2-Math](https://aiwiki.ai/wiki/qwen2_math), which mainly supported English, Qwen2.5-Math solves math problems in both English and Chinese, and it reasons in two modes: plain [chain-of-thought](https://aiwiki.ai/wiki/chain_of_thought) (CoT) and tool-integrated reasoning (TIR), in which the model writes and executes Python code for exact computation and symbolic manipulation.[1]

The accompanying technical report frames the series as an exercise in self-improvement: the earlier Qwen2-Math models generated much of the pretraining and fine-tuning data for their successors, a reward model trained on sampled solutions steered both training and inference, and the loop was iterated.[2] The flagship Qwen2.5-Math-72B-Instruct scored 85.9 on the competition-level [MATH](https://aiwiki.ai/wiki/math) benchmark with greedy chain-of-thought decoding, ahead of the 81.1 the report measured for [GPT-4o](https://aiwiki.ai/wiki/gpt_4o), and reached 92.9 when tool use and reward-model reranking were combined.[1][2]

The models are narrow by design: Alibaba's model cards state that the series mainly supports solving English and Chinese math problems through CoT and TIR and recommend against using it for other tasks.[4][5] That narrowness did not stop the family from having an outsized afterlife. The 1.5B and 7B base models became the foundation of DeepSeek's distilled R1 reasoning models in January 2025, and Qwen2.5-Math-7B became one of the most heavily used testbeds in 2025 research on [reinforcement learning with verifiable rewards](https://aiwiki.ai/wiki/rlvr).[11][12]

## Background

The Qwen team released [Qwen2-Math](https://aiwiki.ai/wiki/qwen2_math), its first math-specialist line, on August 8, 2024. Those models, also in 1.5B, 7B, and 72B sizes with base and instruct variants, were built on the [Qwen2](https://aiwiki.ai/wiki/qwen2) foundation models and pretrained on a mathematics corpus of web text, books, code, exam questions, and synthetic data generated by Qwen2. Alibaba claimed the largest model, Qwen2-Math-72B-Instruct, outperformed GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and Llama-3.1-405B on math benchmarks. The release noted that the models mainly supported English and promised bilingual versions to follow.[3]

Qwen2.5-Math arrived roughly six weeks later, announced in a blog post dated September 19, 2024, with the technical report ("Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement", by An Yang, Beichen Zhang, Binyuan Hui, and colleagues) submitted to arXiv on September 18.[1][2] The specialist models sat alongside the general-purpose [Qwen2.5](https://aiwiki.ai/wiki/qwen2_5) series and its coding sibling [Qwen2.5-Coder](https://aiwiki.ai/wiki/qwen2_5_coder), all announced the same month.

## Model family and licensing

| Model | Parameters | Role |
|---|---|---|
| Qwen2.5-Math-1.5B / 7B / 72B | 1.5B, 7B, 72B | Base models pretrained on the Qwen Math Corpus v2 |
| Qwen2.5-Math-1.5B / 7B / 72B-Instruct | 1.5B, 7B, 72B | Instruction-tuned models supporting CoT and TIR in English and Chinese |
| Qwen2.5-Math-RM-72B | 72B | Reward model for rejection sampling, RL training, and best-of-N inference |

The base models keep a 4K-token context length, reflecting the short-form nature of math problems.[2] The instruct models were initialized from the corresponding math base models, which in turn derive from the general Qwen2.5 base series.[4]

Licensing splits by size, following the pattern of the wider Qwen2.5 release: the 1.5B and 7B instruct models are published under the [Apache License](https://aiwiki.ai/wiki/apache_license) 2.0, while Qwen2.5-Math-72B-Instruct and the RM-72B reward model carry Alibaba's custom "Qwen" license.[4][5][6][7] All of the weights are downloadable from [Hugging Face](https://aiwiki.ai/wiki/hugging_face) and ModelScope.[8]

## Training and self-improvement

The technical report describes self-improvement applied at three stages.[2]

First, in pretraining, Qwen2-Math-72B-Instruct was used to synthesize additional high-quality mathematical data. Together with newly recalled web text, books, and code, this grew the math pretraining set from the 700 billion tokens of Qwen Math Corpus v1 (used for Qwen2-Math) to over 1 trillion tokens in Qwen Math Corpus v2.[1][2] The team decontaminated training data against evaluation sets using 13-gram matching, excluding samples whose longest common subsequence ratio with a test item exceeded 0.6, across benchmarks including [GSM8K](https://aiwiki.ai/wiki/gsm8k), MATH, and AIME 24.[2]

Second, in post-training, a math-specific reward model was trained on solutions sampled from the models themselves, with multiple responses per problem (six in the Qwen2.5 iteration) labeled by answer correctness and ranked with a listwise loss.[2] This reward model then fed back into training in two ways: it selected the best sampled chain-of-thought responses for [supervised fine-tuning](https://aiwiki.ai/wiki/supervised_fine-tuning) data via [rejection sampling](https://aiwiki.ai/wiki/rejection_sampling), and it provided the reward signal for a subsequent [reinforcement learning](https://aiwiki.ai/wiki/reinforcement_learning) stage using [Group Relative Policy Optimization](https://aiwiki.ai/wiki/grpo) (GRPO), a method that avoids the separate value function of PPO.[1][2] As the supervised models improved, the reward model was itself retrained on their outputs, and another round of data polishing followed.[1]

Third, at inference, the reward model reranks sampled solutions (best-of-N), which is where the headline RM@8 and RM@256 numbers come from.[2]

Instruction tuning covered both reasoning modes and both languages: chain-of-thought data in English and Chinese, plus tool-integrated reasoning data in which solutions interleave natural-language reasoning with Python code execution.[1][2]

## Chain-of-thought and tool-integrated reasoning

The two inference modes trade off generality against precision. CoT is ordinary step-by-step natural-language reasoning. TIR has the model emit Python code, run it, and reason over the results, which the Qwen team positions as a fix for the things language models are chronically bad at: exact arithmetic, symbolic manipulation, and algorithmic procedures.[1] The gap is visible on hard benchmarks. On MATH, TIR adds between 1.6 and 4.1 points over CoT depending on model size, and on [AIME](https://aiwiki.ai/wiki/aime) 2024 the 72B instruct model solves 9 of 30 problems with greedy CoT decoding but 12 with TIR.[2] On the easier GSM8K, TIR offers no advantage; the TIR scores land slightly below CoT for all three sizes.[2]

## Benchmark performance

The report evaluates on ten datasets spanning grade school to competition difficulty, including GSM8K, MATH, Minerva Math, [OlympiadBench](https://aiwiki.ai/wiki/olympiadbench), AMC 2023, and AIME 2024 in English, and CMATH and GaoKao exams in Chinese.[2]

Greedy-decoding scores for the instruct models from the technical report:[2]

| Model | GSM8K (CoT) | MATH (CoT) | MATH (TIR) | AIME 2024 (CoT / TIR) | AMC 2023 (CoT / TIR) |
|---|---|---|---|---|---|
| Qwen2.5-Math-1.5B-Instruct | 84.8 | 75.8 | 79.9 | 3/30 / 7/30 | 24/40 / 20/40 |
| Qwen2.5-Math-7B-Instruct | 95.2 | 83.6 | 85.2 | 5/30 / 6/30 | 25/40 / 27/40 |
| Qwen2.5-Math-72B-Instruct | 95.9 | 85.9 | 88.1 | 9/30 / 12/30 | 28/40 / 28/40 |

The release blog and Hugging Face model cards quote slightly different TIR MATH figures (79.7, 85.3, and 87.8 for the three sizes), evidently from a separate evaluation run.[1][4]

Reward-model reranking pushes these numbers higher. With RM@8 in TIR mode, the 72B instruct model reaches 92.9 on MATH; with RM@256 in CoT mode, the 1.5B instruct model solves 29 of 40 AMC 2023 problems; and with reward-model assistance the 7B instruct model solved up to 21 of 30 AIME 2024 problems.[1] The blog argues the mid-size model had effectively overtaken the previous flagship: Qwen2.5-Math-7B-Instruct surpasses Qwen2-Math-72B-Instruct in performance.[1][8]

The base models improved over their Qwen2-Math counterparts by 5.4, 5.0, and 6.3 points on MATH, and the new bilingual training showed up as gains of 3.4, 12.2, and 19.8 points on GaoKao Math QA.[1] One quirk shows up in the base-model tables: the 7B slightly beats the 72B on GSM8K (91.6 versus 90.8, few-shot), which the report reads as evidence that better data and training strategy can partly compensate for parameter count.[2]

## Reward models and process supervision

Qwen2.5-Math-RM-72B, released with the series, is an outcome-style reward model that scores whole solutions, used for rejection sampling during data construction, as the RL reward signal, and for best-of-N selection at inference.[7]

In January 2025 the team extended this to step-level supervision with two process reward models, Qwen2.5-Math-PRM-7B and Qwen2.5-Math-PRM-72B, released alongside the paper "The Lessons of Developing Process Reward Models in Mathematical Reasoning".[9][10] The paper is partly a negative result: it reports that the popular approach of synthesizing process labels through Monte Carlo estimation yields worse PRMs than human annotation or LLM-as-a-judge labeling, and that best-of-N evaluations are biased toward models that tolerate flawed reasoning so long as the final answer is right. The authors propose a consensus-filtering mechanism combining Monte Carlo estimation with LLM-as-a-judge and evaluate step-level error identification on [ProcessBench](https://aiwiki.ai/wiki/processbench), a benchmark the team had introduced the previous month.[9][10] The PRM-72B model, fine-tuned from Qwen2.5-Math-72B-Instruct, outputs a probability-style reward between 0 and 1 after each reasoning step.[10]

## Use as a research base

Qwen2.5-Math's most visible legacy is as raw material for other people's reasoning models. When [DeepSeek](https://aiwiki.ai/wiki/deepseek) released [DeepSeek-R1](https://aiwiki.ai/wiki/deepseek_r1) in January 2025, the two smallest of its [distilled](https://aiwiki.ai/wiki/deepseek_r1_distill) variants, DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B, were fine-tuned from Qwen2.5-Math-1.5B and Qwen2.5-Math-7B respectively.[11]

Through 2025, Qwen2.5-Math-7B also became a default starting checkpoint for academic work on reinforcement learning with verifiable rewards, which made its idiosyncrasies a research topic in their own right. The "Spurious Rewards" study (Shao et al., June 2025) found that GRPO training with randomly assigned rewards improved Qwen2.5-Math-7B's [MATH-500](https://aiwiki.ai/wiki/math_500) accuracy by 21.4 percentage points, close to the 29.1-point gain from ground-truth rewards, while the same spurious signals produced little or no gain for Llama 3 or OLMo 2 models.[12] The authors concluded that RL methods should be validated across diverse model families rather than a single de facto choice, since large gains can arise on Qwen models even from random rewards that reflect no genuine capability improvement.[12]

Within Alibaba's own lineup, the specialist line fed the generalists and then ended. The [Qwen3](https://aiwiki.ai/wiki/qwen_3) release in April 2025 used Qwen2.5-Math (with Qwen2.5-Coder for code) to synthesize the mathematical portion of its pretraining data, and shipped competition-math ability inside general models through an optional step-by-step thinking mode rather than as a separate math series.[13] No Qwen3-branded math-specialist models had been announced as of mid-2026, leaving Qwen2.5-Math as the last dedicated math line from the team.

## Limitations

The models are deliberately narrow. Alibaba's own documentation recommends against using them for anything other than English and Chinese math problem solving.[4][5] The 4K context window, adequate for competition problems, rules out long-document use.[2] TIR performance depends on a Python execution environment, and the strongest published numbers rely on reward-model reranking over many samples, which multiplies inference cost.[1][2] Finally, the RLVR research discussed above suggests caution when interpreting downstream results built on these checkpoints: some of what looks like learning in fine-tuned derivatives may be elicitation of behaviors already present in the base models.[12]

## See also

- [Qwen2.5](https://aiwiki.ai/wiki/qwen2_5)
- [Qwen2-Math](https://aiwiki.ai/wiki/qwen2_math)
- [Qwen2.5-Coder](https://aiwiki.ai/wiki/qwen2_5_coder)
- [Mathematical reasoning](https://aiwiki.ai/wiki/mathematical_reasoning)
- [MATH benchmark](https://aiwiki.ai/wiki/math)
- [DeepSeekMath](https://aiwiki.ai/wiki/deepseek_math)

## References

1. Qwen Team. "Qwen2.5-Math: The world's leading open-sourced mathematical LLMs." Qwen blog, September 19, 2024. https://qwenlm.github.io/blog/qwen2.5-math/
2. Yang, An; Zhang, Beichen; Hui, Binyuan; et al. "Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement." arXiv:2409.12122, September 18, 2024. https://arxiv.org/abs/2409.12122
3. Qwen Team. "Introducing Qwen2-Math." Qwen blog, August 8, 2024. https://qwenlm.github.io/blog/qwen2-math/
4. Hugging Face model card: Qwen/Qwen2.5-Math-72B-Instruct. https://huggingface.co/Qwen/Qwen2.5-Math-72B-Instruct
5. Hugging Face model card: Qwen/Qwen2.5-Math-7B-Instruct. https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct
6. Hugging Face model card: Qwen/Qwen2.5-Math-1.5B-Instruct. https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct
7. Hugging Face model card: Qwen/Qwen2.5-Math-RM-72B. https://huggingface.co/Qwen/Qwen2.5-Math-RM-72B
8. QwenLM/Qwen2.5-Math repository, GitHub. https://github.com/QwenLM/Qwen2.5-Math
9. Zhang, Zhenru; Zheng, Chujie; Wu, Yangzhen; et al. "The Lessons of Developing Process Reward Models in Mathematical Reasoning." arXiv:2501.07301, January 13, 2025. https://arxiv.org/abs/2501.07301
10. Hugging Face model card: Qwen/Qwen2.5-Math-PRM-72B. https://huggingface.co/Qwen/Qwen2.5-Math-PRM-72B
11. Hugging Face model card: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
12. Shao, Rulin; Li, Shuyue Stella; Xin, Rui; et al. "Spurious Rewards: Rethinking Training Signals in RLVR." arXiv:2506.10947, June 12, 2025. https://arxiv.org/abs/2506.10947
13. Qwen Team. "Qwen3: Think Deeper, Act Faster." Qwen blog, April 29, 2025. https://qwenlm.github.io/blog/qwen3/