Absolute Zero Reasoner

Absolute Zero Reasoner (AZR) is a research system for post-training a pretrained language model through self-play on machine-generated Python reasoning tasks. One policy alternates between proposing tasks and solving them, while a code executor checks task validity and answer correctness. Andrew Zhao and nine coauthors from Tsinghua University, the Beijing Institute for General Artificial Intelligence, and Pennsylvania State University introduced the system in "Absolute Zero: Reinforced Self-play Reasoning with Zero Data," published in the NeurIPS 2025 main conference track ^[1].

The phrase "zero data" has a specific boundary. During AZR's reinforcement learning stage, the researchers did not supply an external collection of human-written or distilled questions, answers, or reasoning traces. AZR was not trained from random initialization without data: each experiment began with a pretrained Qwen2.5 or Llama 3.1 model ^[1]. For context, the Qwen2.5 technical report says that the model family was pretrained on up to 18 trillion tokens ^[6]. AZR is therefore a zero-external-task-data method for one training stage, not a data-free model lifecycle.

Relationship to other zero-data methods

Reinforcement learning with verifiable rewards normally trains a model on a fixed collection of questions for which an automatic checker can judge the final answer, rather than relying on a learned reward model. DeepSeek-R1-Zero, for example, omitted supervised fine-tuning before reinforcement learning but still used curated reasoning questions and optimized with GRPO ^[10]. Earlier self-training methods such as STaR generated rationales for an existing question set; they did not generate the questions themselves ^[12]. AZR changes the source of those questions by generating the task distribution on which it is trained ^[1].

This setup resembles self-play because the same parameterized policy performs two roles:

• The proposer generates a candidate task. Its reward depends on how often the current solver can answer that task.
• The solver attempts the generated task. It receives a binary correctness reward from executable verification.

The analogy to systems such as AlphaZero is limited. Board-game self-play operates under a fixed game state and fixed legal moves. AZR searches over model-generated programs and inputs, and its practical task space is bounded by the Python environment, filters, and capabilities of the starting model ^[1].

Task representation

AZR represents a valid task as a Python program (p), an input (i), and the executed output (o), such that (p(i)=o). The paper groups its tasks into three modes ^[1]:

Scroll sideways for more →

For abduction, the solver does not have to recover the proposer's original input because a program can map multiple inputs to the same output. For induction, withholding part of the generated test set is intended to discourage a program that merely hard-codes the visible examples ^[1].

The three labels describe code-based task formats, not proof that the system acquired every form of deduction, abduction, or induction studied in logic or cognitive science. All three reduce to properties that the released Python evaluator can execute and compare.

Self-play and buffer initialization

The seed procedure is more substantial than a single hand-written training example. At time zero, an empty seed buffer causes the prompt to fall back to a trivial identity-function triplet. The pretrained model then proposes program-input pairs, which are filtered and executed. With the paper's batch size (B=64) and seed factor (S=4), the accepted seed buffers are filled to (B \times S=256) entries; a comparable induction buffer is generated from those programs. No policy update occurs during this seeding phase ^[1].

During self-play, deduction and abduction proposals are conditioned on six previously accepted triplets. An induction proposal begins from a program sampled from the deduction and abduction buffers and generates matching inputs plus a description. Accepted proposals are appended to task-specific buffers. If filtering leaves fewer than a full solver batch, the implementation samples older validated tasks from the corresponding buffer to fill it ^[1].

The paper's validator checks whether proposed code parses, executes, avoids a listed set of modules or operations, and appears deterministic. The determinism test is only two independent executions, so it can miss intermittent or state-dependent behavior. The authors also state that the bundled raw Python executor is not secure enough for production and recommend an isolated execution service instead ^[1][2].

Rewards and optimization

For each proposed task, the system samples eight solver attempts and computes their mean binary success rate (s). The proposer reward in the published method is:

• (0) when (s=0);
• (1-s) when (s>0).

Consequently, the largest possible proposer reward with eight attempts, 0.875, occurs when exactly one attempt succeeds. Tasks that fail all attempts or pass all attempts both receive zero. This is not a symmetric reward centered on a 50 percent solve rate, although the paper describes the intent as avoiding both impossible and trivial tasks ^[1].

The solver receives 1 for a correct value and 0 otherwise. A composite formatting rule retains the role reward for a valid response, assigns -0.5 to an incorrect but well-formed response, and -1 to a formatting failure. A proposer output counts as well formed only if it also passes parsing, execution, safety, and determinism filters ^[1].

AZR uses Task-Relative REINFORCE++ (TRR++). Instead of normalizing rewards against one global batch baseline, it separately estimates the mean and standard deviation for each of six task-role combinations: three task modes times proposer or solver. The published runs used a clipped PPO objective, one policy epoch per batch, a learning rate of (10^{-6}), 500 training steps, an entropy coefficient of 0.001, and no KL loss or KL reward. The paper reports that each experiment ran for roughly three to five days on clusters of NVIDIA A800 GPUs ^[1].

Evaluation protocol

The main out-of-distribution evaluation used three code-generation suites and six mathematical-reasoning suites. HumanEval+ and MBPP+ are extended-test versions supplied by EvalPlus ^[7]. LiveCodeBench Generation covered problems released from May 2023 through February 2025 ^[8]. The mathematics group comprised AIME 2024, AIME 2025, AMC 2023, MATH-500, Minerva Math, and OlympiadBench. The paper also treated CRUXEval input prediction, CRUXEval output prediction, and LiveCodeBench code execution as in-distribution evaluations because they resemble AZR's generated code tasks ^[1][9].

The principal result table used greedy decoding. Its code average is the mean of HumanEval+, MBPP+, and LiveCodeBench Generation; its mathematics average is the mean of the six math suites; and its overall average is the mean of those two group averages. These are benchmark-author results under that fixed protocol, not a continuously updated independent leaderboard ^[1].

Scroll sideways for more →

For AZR-Coder-7B, the paper reported the following underlying scores ^[1]:

Scroll sideways for more →

The change column above is calculated from the two displayed score columns. NeurIPS Table 1 prints 72.6 for AZR-Coder-7B on MATH-500 and 54.0 for its listed starting model, but annotates the 72.6 value as a +22.6 gain. Those printed values differ by 18.6, so the table above reports +18.6 and treats the paper's gain annotation as internally inconsistent ^[1].

The paper's authors described AZR-Coder-7B as the best overall result among the selected "zero-style" 7B comparisons in their table. On the defined overall average it exceeded the next row by 1.8 points, and on the code average by 0.3 points ^[1]. That statement is specific to the compared models, benchmark versions, and evaluation settings. It should not be read as a timeless claim about all reasoning models.

Scaling and ablations

Across the three tested Qwen2.5-Coder sizes, the reported overall gain increased with model size: 35.0 to 40.7 for 3B, 40.2 to 50.4 for 7B, and 40.1 to 53.3 for 14B. These three observations suggest that the method benefited more capable starting models in the reported setup, but they do not establish a general scaling law. On Llama 3.1 8B, AZR improved the overall average from 16.0 to 19.2, while the same-base SimpleRL comparison in the paper reached 20.5 ^[1].

The authors also trained ablations on Qwen2.5-7B base ^[1]:

Scroll sideways for more →

Those comparisons support contributions from the task mixture, generated references, and proposer updates within this one 7B experiment. The paper does not report repeated-run confidence intervals, so the table does not quantify run-to-run uncertainty.

Later analysis

A later study by Justin Yang Chae, Md Tanvirul Alam, and Nidhi Rastogi analyzed AZR training with Qwen2.5-Coder 3B and 7B models ^[5]. It found that:

• AZR improved performance at small sampling budgets, while the starting model sometimes regained an advantage at large (k). The measured large-(k) decline was not statistically significant.
• Generated questions became harder over training, and the 7B model's response length increased as measured solve rates fell.
• Policy entropy still declined during self-play. Proposer entropy remained higher than solver entropy, and freezing proposer updates slowed the decline in the 3B experiment.
• AZR-Coder-7B had about 45.7 percent parameter-update sparsity in the study's checkpoint comparison, intermediate between its chosen RLVR and supervised-fine-tuning checkpoints.
• Replacing the original proposer reward with one maximized at a 50 percent solve rate had little effect on entropy and reduced final validation accuracy by 2 percentage points in that experiment.

The follow-up argues that self-play cannot assign probability to solutions outside the starting model's support. Its empirical scope was AZR and two model sizes, so its findings should not automatically be generalized to every self-play method or larger model ^[5].

Release and reproducibility

The official repository contains training and evaluation code under the MIT License. It is built on veRL, uses vLLM for rollout generation, and includes an executor adapted from the QwQ project ^[2]. The repository distinguishes two reproducibility targets:

• The fixed paper branch at commit 41ed983 contains the code associated with the published experiments ^[3].
• The later master branch migrated to a newer veRL version and, as of the research cutoff, its README described that path as under testing. Subsequent commits included fixes for accuracy-sampling or answer-cache behavior and for the stateful data loader ^[2][11].

This branch distinction matters when comparing a reproduction with the NeurIPS tables. It does not by itself show that the published results are wrong.

The authors' official Hugging Face collection lists five released Qwen2.5 checkpoints: Coder 3B, Coder 7B, Coder 14B, Base 7B, and Base 14B ^[4]. It does not list the experimental Llama 3.1 8B model. The repository's MIT file clearly licenses the code, but the five checkpoint pages do not provide model cards or an explicit weight-license field. The code license should therefore not be assumed to settle the licensing terms for the model weights.

Limitations and safety

AZR's strongest evidence concerns one verifiable environment and one benchmark suite. Important boundaries include:

• Task domain: the released method generates Python-executable code tasks. The paper proposes extending the idea to other environments but does not demonstrate open-ended self-improvement on the web, formal mathematics, simulators, or the physical world ^[1].
• Verifier coverage: a forbidden-module list and two executions are incomplete substitutes for secure sandboxing and formal determinism. Incorrectly accepted proposals can contaminate rewards, and production use requires stronger isolation ^[1][2].
• Starting-model dependence: AZR inherits capabilities and data exposure from its pretrained base model. The evaluation does not audit whether Qwen2.5 pretraining included material related to every benchmark ^[1][6].
• Evaluation evidence: the main scores are reported by the method's authors. No independent full reproduction of the published table was located by July 28, 2026, and the paper does not provide repeated-run uncertainty for the main comparisons.
• Safety evidence: the paper displays one Llama 3.1 8B reasoning trace with adversarial language and calls it an "uh-oh moment." This is a concrete example that motivates oversight, not a prevalence estimate or proof that AZR generally develops hostile objectives ^[1].
• Compute and implementation sensitivity: the experiments required multi-GPU clusters for several days, and the repository's original-results branch differs from its later development branch ^[1][2][3].

AZR is evidence that a pretrained model can improve selected code and math benchmark scores during self-generated, executable-task training without an external task dataset. It is not evidence that training data are unnecessary, that the improvement is unbounded, or that a code verifier can supervise arbitrary real-world reasoning.

References

1. ^Zhao, Andrew; Wu, Yiran; Yue, Yang; Wu, Tong; Xu, Quentin; Lin, Matthieu; Wang, Shenzhi; Wu, Qingyun; Zheng, Zilong; Huang, Gao. "Absolute Zero: Reinforced Self-play Reasoning with Zero Data." Advances in Neural Information Processing Systems 38, NeurIPS 2025 Main Conference Track. papers.nips.cc/...68ed48042d49-Abstract-Conference
2. ^LeapLabTHU. "Absolute Zero Reasoner," official repository, `master` snapshot at commit `484afa4`. github.com/...fa480c8f6fd77faa3d35451f24f287f58ee1
3. ^LeapLabTHU. "Absolute Zero Reasoner," original-results `paper` branch snapshot at commit `41ed983`. github.com/...983cdf541cfcd2f963f33c055d50074f3c90
4. ^Zhao, Andrew. "Absolute Zero Reasoner," official model collection, Hugging Face. huggingface.co/...absolute-zero-reasoner
5. ^Chae, Justin Yang; Alam, Md Tanvirul; Rastogi, Nidhi. "Towards Understanding Self-play for LLM Reasoning." arXiv:2510.27072, 2025. arxiv.org/...2510.27072
6. ^Yang, An et al. "Qwen2.5 Technical Report." arXiv:2412.15115, 2024. arxiv.org/...2412.15115
7. ^Liu, Jiawei; Xia, Chunqiu Steven; Wang, Yuyao; Zhang, Lingming. "Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation." arXiv:2305.01210. arxiv.org/...2305.01210
8. ^Jain, Naman et al. "LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code." ICLR 2025. proceedings.iclr.cc/...ed43767-Abstract-Conference
9. ^Gu, Alex; Roziere, Baptiste; Leather, Hugh James; Solar-Lezama, Armando; Synnaeve, Gabriel; Wang, Sida. "CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution." Proceedings of the 41st International Conference on Machine Learning, PMLR 235, 2024. proceedings.mlr.press/...gu24c
10. ^DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." arXiv:2501.12948, 2025. arxiv.org/...2501.12948
11. ^LeapLabTHU. "Bug fix for getting identical outputs for acc estimation, also answer caching doesnt include None rewards," commit `91f0741`, August 23, 2025. github.com/...741925c12d79c069e99b1bf1216ca176628d
12. ^Zelikman, Eric; Wu, Yuhuai; Mu, Jesse; Goodman, Noah. "STaR: Bootstrapping Reasoning With Reasoning." Advances in Neural Information Processing Systems 35, 2022. proceedings.neurips.cc/...e9a5-Abstract-Conference