CodeMidas

CodeMidas is an agentic data pipeline from Xiaomi's LLM Core team that turns functionality already implemented in open-source codebases into executable reinforcement learning environments for coding agents. It is described in the paper "CodeMidas: Scaling Agentic Coding RL Environments from Code Itself," submitted to arXiv on 18 September 2026 (arXiv:2609.22068).^[1] Unlike pipelines that mine GitHub issues, pull requests or commits, CodeMidas uses source code as its only task-specific input: agents pick a working feature, delete its core implementation, write a behavioral task statement, and build hidden tests whose expected values come from running the original code.^[1] The resulting dataset has 5,545 training tasks drawn from 3,185 open-source codebases in 23 programming languages and 15 technical domains. Training Xiaomi's MiMo-V2.5 on these tasks with GRPO improved its scores on all five external benchmarks the authors tested, including a rise on DeepSWE v1.1 from 10.0% to 21.7%.^[1] Xiaomi's later MiMo-V2.6 technical report names CodeMidas as the tool behind one of the task-synthesis pathways used for that model's coding RL.^[3]

The name refers to the Midas touch: the paper says CodeMidas "applies a Midas touch" to existing code, "turning existing code into RL environments."^[1]

Authors

The paper lists 19 authors. Bowen Ye is first author (the work was done during an internship at Xiaomi), and Tong Yang and Fuli Luo are co-corresponding authors.^[1]

Several authors hold two affiliations, so some names appear in more than one row.^[1] On Hugging Face, the paper is listed under the Xiaomi MiMo organization and was submitted to the site's daily papers feed by Lei Li on 21 September 2026.^[2]

Motivation

Training a coding agent with RL needs a large supply of tasks, each with a verifier that reliably tells correct work from incorrect work. The paper argues that most existing environment pipelines are tied to development artifacts. Some derive task statements from issues, pull requests or commits; others synthesize bugs or development tasks around a project's existing tests, or use existing documentation to specify the requested functionality. Each approach can only produce tasks where those records exist.^[1]

CodeMidas starts from a different observation: any implemented feature in a working codebase already supplies both the basis for a task and a candidate solution. Its public interfaces and observable behavior define what an agent should build, executing the original code supplies expected test values, and the rest of the repository can be kept as a realistic starting point with its real structure and dependencies.^[1] The paper stresses two requirements for such tasks: the statement must make the required behavior explicit while leaving internal implementation choices open, and the tests must reject incorrect solutions while still accepting alternative correct implementations.^[1]

Pipeline

Each CodeMidas task consists of a task statement, a containerized development environment, and a hidden executable verifier. The solver sees the statement and the adapted codebase with its dependencies. The verifier stays outside the solver's environment and is injected only at grading time, where it returns a binary reward.^[1] The paper organizes construction into four modules.

The paper's overview figure shows the candidate pool shrinking through these modules, with retained counts of 22,575, 16,027, 12,746, 11,930, 8,173 and finally 5,545 tasks.^[1]

Task design

Supported interfaces are command-line tools, pure library functions and stateful library APIs, graded respectively through process outputs, return values, and state changes across calls.^[1] The task statement and the code boundary are revised together so that shared components and project context are preserved. The statement defines inputs, observable behavior and the required public interfaces; solvers are free to choose their own internal helpers and algorithms.^[1]

Test construction

Tests take the form of command executions for CLI tools, input-output cases for pure functions, and sequences of calls for stateful APIs, including ordering and cleanup behavior where the statement specifies it. Each test records which requirement it covers.^[1] For outputs fixed by the statement, expected values come from executing the reference code. For aspects the statement leaves open, assertions check only the stated constraints: the paper's example is a test that enforces a required exception type without fixing the wording of the error message.^[1]

A reviewing agent then examines every assertion for restrictions not supported by the statement, such as exact wording, incidental ordering or internal structure, and replaces them with behavioral checks. A task is rejected if an assertion depends on a private symbol and has no behavioral substitute. The revised tests are rerun on the reference solution before being frozen for grading.^[1]

Environment preparation and execution consistency

Cleanup removes compiled outputs, cached copies of the source, files left behind by the construction agents, and the project's original tests for the target functionality, while keeping the packages, fixtures and build wrappers needed to build a completed implementation.^[1]

In the execution-consistency check, each task runs under the training runtime settings in six fresh containers: two with the starting codebase and four with the reference solution in place. Both starting-state runs must fail and all four reference runs must pass. The paper says the repetitions both confirm the expected fail-to-pass transition and screen out unstable execution outcomes.^[1]

Post-rollout filtering

The last module uses agent attempts rather than static checks.

• Leakage filtering. In adversarial rollouts, an agent tries to recover a solution without doing the intended development work, searching the whole solver-visible environment, including compiled artifacts, caches, files left by construction agents, and installed copies of the target project. It logs the commands and outputs behind each suspected exploit, and a separate review checks that evidence against the reference solution and verifier. Tasks are rejected if the review confirms that leaked material can bypass the work.^[1] The paper calls this exploitable leakage; it is a route to reward hacking, although the paper does not use that term.
• Agreement on agent solutions. A coding agent attempts each task four times. A reviewing agent reads the trajectories, submitted code and test outputs alongside the statement, verifier and reference solution, and judges whether each implementation satisfies the statement. It flags a false positive when an implementation it judges incorrect passes the tests, and a false negative when one it judges correct fails. Tasks with identified verifier defects are rejected.^[1]
• Rollout outcome filtering. A frontier model, which the paper does not name, makes several attempts per task. Tasks where every attempt passes or every attempt fails are dropped, because either outcome may reflect difficulty or a remaining defect such as weak tests or a requirement missing from the statement, and the outcome alone does not reveal which. Only tasks with both successful and failed attempts under that model and budget are kept.^[1]

Dataset

The final training set contains 5,545 tasks from 3,185 codebases. Each task inherits the primary language label of its codebase; the ten most common languages cover 5,445 tasks (98.2%) and the full set spans 23 languages.^[1]

Domain labels come from the codebases. Systems software (17.42%), web (14.61%) and developer tools (13.56%) are the largest domains, together 45.6% of tasks. They are followed by AI/ML (9.11%), data science (7.57%), multimedia (7.14%), specialized (6.89%), cloud/DevOps (3.95%), blockchain (3.55%), desktop (3.28%), security (3.25%), game development (3.08%), hardware (2.67%), automation (2.02%) and mobile (1.57%), with 18 unlabeled tasks (0.32%) grouped as "Other."^[1]

Reference solutions are sizeable. Counting every source line added or deleted in the reference patch, including comments and blank lines, the median is 142 lines, with an interquartile range of 66 to 305 lines, and 65.9% of reference patches touch at least two source files.^[1]

Training setup

The authors trained MiMo-V2.5 on the 5,545 tasks with GRPO, the group-relative policy optimization method introduced with DeepSeekMath, using the binary verifier outcome as the reward and no reward model or learned verifier.^[1] Appendix A lists the configuration:

Evaluation used five external benchmarks with their official task sets: SWE-bench Pro, DeepSWE v1.1, ProgramBench, RepoZero C2Rust and Terminal-Bench v2.1. The authors also held out CodeMidas Val, 200 randomly sampled CodeMidas tasks separate from the training set, scored over three attempts per task. They state that the training set is disjoint from CodeMidas Val and from all five external benchmark task sets. ProgramBench is scored by "Almost Solved," the share of tasks passing at least 95% of their tests; the other benchmarks report pass rate.^[1]

Results

All figures below are the authors' own measurements of MiMo-V2.5 before and after CodeMidas RL, under identical evaluation settings. Gains are absolute percentage points.^[1]

The abstract writes these gains with percent signs ("DeepSWE + 11.7%", "ProgramBench +17%"), but the body and Figure 5 make clear they are absolute differences in percentage points, not relative improvements.^[1] SWE-bench Verified is not among the five benchmarks reported.

On CodeMidas Val, pass rate rose from 35.0% to 44.7% during training, staying roughly 8 to 10 points above the initial policy at evaluated checkpoints from step 40 onward. The gains came with longer trajectories, which the authors read as greater use of the available interaction budget.^[1]

Scale and quality ablation

To separate the effect of task count from the effect of filtering, the authors trained on random 1k and 3k subsets of the filtered dataset, on the full 5,545-task set, and on a "vanilla" sample of about 8,000 tasks drawn before filtering. The vanilla tasks each had a statement, environment and verifier but skipped environment cleaning, the execution-consistency check and all three post-rollout filters. All four runs used the same training configuration and checkpoint range.^[1]

Scores rise with the size of the filtered pool on all three evaluations. The full filtered set beats the vanilla 8k sample by 0.59, 4.59 and 4.49 points on SWE-bench Pro, DeepSWE and CodeMidas Val, and even the 3k filtered subset beats the vanilla sample on all three.^[1] The authors conclude that cleaning, execution checks and post-rollout filtering add value even with fewer tasks. On CodeMidas Val learning curves, the full dataset led at every evaluated checkpoint from step 40 to step 70.^[1]

Behavioral analysis

The paper also measures how agent behavior changed during RL, using three metrics defined in its Appendix B: codebase exploration (distinct read or search requests before the first edit), code drafting (the share of 16-character fragments in Write/Edit payloads that already appeared in the preceding reasoning) and self-verification (distinct verification commands run after the final edit).^[1]

Within the same task and checkpoint on CodeMidas Val, rollouts in which the agent wrote and ran its own checks had a mean pass rate 4.2 points higher than rollouts without them (95% confidence interval 1.8 to 6.6). The corresponding differences for above-median exploration (+0.7 points) and drafting (+1.95 points) had confidence intervals that include zero.^[1]

The authors report that some of these shifts carried over to held-out benchmarks. Comparing the first and last three checkpoints, exploration rose on SWE-bench Pro (23.1 to 35.5 calls), ProgramBench (55.7 to 83.6) and Terminal-Bench v2.1 (11.9 to 16.8), while changes in verification diversity varied by benchmark. Interaction length moved in opposite directions by task type: mean turns rose from 37.3 to 50.1 on SWE-bench Pro but fell from 155.1 to 122.8 on ProgramBench.^[1]

Comparison with other environment pipelines

The paper's Table 1 compares the task-specific inputs required by representative environment-construction pipelines, including SWE-rebench V2, R2E-Gym and SWE-smith. In the table, "not required" means the pipeline works without that input, "partial" means part of the pipeline requires it, and "required" means it is needed.^[1]

The related-work section credits SWE-bench with establishing repository-level issue resolution as an execution-based evaluation setting, and describes R2E-Gym as generating tests and statements from commits, SWE-smith as synthesizing code changes that break existing tests, and R2E and MindForge as starting from docstrings or documentation.^[1] On the reward side, the authors contrast their plain execution rewards with approaches such as SWE-RL's reference-patch similarity, process reward models and rubric-based verifiers, and cite work showing that original test suites can accept incorrect patches as the motivation for their verifier checks.^[1]

Use in MiMo-V2.6

Xiaomi's technical report "MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement" describes several pathways for building coding RL tasks: GitHub pull requests and issues, development requests from Xiaomi employees, specification-driven tasks, source-code-driven synthesis, and long-horizon software-engineering tasks, supplemented by filtered public datasets and licensed data vendors. For the source-code-driven pathway, the report says "we use CodeMidas (Ye et al., 2026) to derive tasks from functionality implemented in existing codebases," noting that it works "without requiring issues, pull requests, or other development artifacts."^[3]

The two documents describe related but distinct checks. The CodeMidas paper's own execution-consistency test uses six containers (two failing starting states, four passing reference runs).^[1] The MiMo-V2.6 report describes, for its coding corpus as a whole, eight reruns of fail-to-pass and pass-to-pass tests for tasks with a reference patch, plus a four-rollout audit in which an auditing agent flags potential false positives and false negatives.^[3] The CodeMidas experiments themselves were run on MiMo-V2.5, not on MiMo-V2.6.^[1] The Hugging Face paper page lists Xiaomi's live RL-metrics dashboard for the MiMo-V2.6-Pro and MiMo-V2.6-Flash training runs as the project page.^[2][5]

Release status and reception

The arXiv v1 paper does not link a code or data repository, and on 23 September 2026 the Hugging Face paper page listed no linked models or datasets.^[1][2] The MiMo-V2.6 report says Xiaomi released RL environments including about 3,000 software-engineering tasks with executable tests, but it does not say which synthesis pathways those tasks came from.^[3] A 22 September 2026 article by the Chinese AI outlet Zhixingxing, published on Sohu, reported that second author Lei Li had written on Xiaohongshu that the team had open-sourced the data pipeline after the MiMo RL livestream ended.^[4]

Lei Li announced the paper on X on 21 September 2026.^[6] The same day, Elie Bakouch, whose X profile describes him as a researcher at Prime Intellect, called it a "very nice paper by Xiaomi Mimo detailing their data factory to get RL environments from open repositories," highlighting its use of "agents in the loop" for task creation, robustness testing ("they let an agent try to cheat") and difficulty selection ("they look at pass@n").^[7] The paper itself describes difficulty filtering as keeping only tasks with both passing and failing attempts from an unnamed frontier model, without stating a specific n.^[1]

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34 ^35 ^36 ^37 ^38 ^39 ^40 ^41 ^42 ^43 ^44 ^45 ^46B. Ye, L. Li, S. Li, Z. Yue, L. Zhang, H. Lv, Y. Liu, W. Ma, H. Tian, R. Li, J. Dong, Y. Zhao, X. Deng, H. Zhang, L. Zhao, Q. Liu, L. Kong, T. Yang and F. Luo, "CodeMidas: Scaling Agentic Coding RL Environments from Code Itself," arXiv:2609.22068, submitted 18 September 2026. arxiv.org/...2609.22068
2. ^1 ^2 ^3Hugging Face Papers, "CodeMidas: Scaling Agentic Coding RL Environments from Code Itself," accessed 23 September 2026. huggingface.co/...2609.22068
3. ^1 ^2 ^3 ^4LLM-Core Xiaomi, "MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement," technical report, September 2026. huggingface.co/...MiMo_V2_6_technical_report.pdf
4. ^Zhixingxing, "小米MiMo RL直播收官，罗福莉团队开源Coding Agent数据管线！" (Xiaomi MiMo RL livestream wraps up; Luo Fuli's team open-sources coding-agent data pipeline), Sohu, 22 September 2026. sohu.com/...1079462415_122980439
5. ^Xiaomi MiMo, "mimo-v2.6 RL" training-metrics dashboard, accessed 23 September 2026. mimo.xiaomi.com/rl
6. ^Lei Li (@_TobiasLee), post announcing CodeMidas, X, 21 September 2026. x.com/...2102031675916169648
7. ^Elie Bakouch (@eliebakouch), post on CodeMidas, X, 21 September 2026. x.com/...2102045988143710664