# AIRA (Meta AI Research Agents)

> Source: https://aiwiki.ai/wiki/aira
> Updated: 2026-09-07
> Fact-checked: 2026-09-07
> Categories: AI Agents, AI Research, Meta AI
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "AIRA (Meta AI Research Agents)." aiwiki.ai, 7 Sept 2026. https://aiwiki.ai/wiki/aira
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**AIRA** (AI Research Agent) is the name [Meta FAIR](https://aiwiki.ai/wiki/meta_fair) uses for its line of autonomous machine learning research agents and for the software that runs them. The line began in July 2025 with the AIRA-dojo framework and a paper on search policies and operators for [MLE-bench](https://aiwiki.ai/wiki/mle_bench), continued in March 2026 with AIRA₂, an asynchronous multi-GPU evolutionary agent, and reached a third generation, AIRA₃, which Meta announced on September 5, 2026 had placed eighth of roughly 4,000 teams and won a gold medal in a live [Kaggle](https://aiwiki.ai/wiki/kaggle) competition run by NVIDIA.[1][3][9] The same FAIR group has used the AIRA-dojo harness for related work on neural architecture discovery (AIRA-Compose and AIRA-Design), on a research-agent benchmark (AIRS-Bench), and on preference models that decide which experiments an agent should run.[5][6][8]

The agents share one basic design: a search process that keeps a graph or population of candidate solutions (usually Python code that trains a model), applies large language model operators to produce new candidates, and scores each candidate by executing it in an isolated container. What changed across the generations is the throughput of that loop, the reliability of the evaluation signal that steers it, and the autonomy of the operators. Everything published so far about the design comes from the first two papers; as of September 6, 2026, Meta had not published a paper, blog post, or code for AIRA₃, and the only first-party description of it is a three-paragraph post and a nine-second logo video.[9]

## Name and lineage

FAIR writes the generation number as a subscript (AIRA₂, AIRA₃) in its papers and posts; this article uses that form in prose and plain digits in headings. "AIRA-dojo" is the framework; "AIRA" on its own in the 2025 paper denotes agents that use FAIR's operator set inside that framework, so "AIRA-greedy" or "AIRA-mcts" name an operator set paired with a search policy. In the 2026 paper "AIRA-dojo" is also used as shorthand for the 2025 agent when it appears as a baseline.[1][3]

| Date | Item | What it is | Source |
| --- | --- | --- | --- |
| Jun 16, 2025 | facebookresearch/aira-dojo repository created | Framework for developing and evaluating AI research agents; CC BY-NC 4.0 code license; implements the MLE-bench task and the paper's agents | [2] |
| Jul 3, 2025 | "AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench" (arXiv 2507.02554; v2 Nov 4, 2025) | 25 authors led by Edan Toledo, Karen Hambardzumyan, Martin Josifoski; formalizes agents as search policy plus operators; MLE-bench lite medal rate 39.6% to 47.7% | [1] |
| Nov 19, 2025 | "What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity" (arXiv 2511.15593) | Audran-Reiss et al.; analysis of MLE-bench trajectories showing higher-performing agents have more ideation diversity | [7] |
| Feb 6, 2026 | "AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents" (arXiv 2602.06855) | Lupidi et al.; 20 tasks sourced from recent ML papers; task definitions and evaluation code open-sourced | [6] |
| Mar 27, 2026 | "AIRA₂: Overcoming Bottlenecks in AI Research Agents" (arXiv 2603.26499; v2 Apr 13, 2026) | 25 authors led by Hambardzumyan, Nicolas Baldwin, Toledo, Rishi Hazra, Michael Kuchnik; asynchronous 8-GPU worker pool, Hidden Consistent Evaluation, ReAct agents; MLE-bench-30 Percentile Rank 81.5% at 24 h | [3][4] |
| May 15, 2026 | "Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design" (arXiv 2605.15871) | Alberto Pepe et al., 8 authors all at FAIR; agent-driven architecture search on top of AIRA-dojo and the AIRS-Bench task format | [5] |
| Jun 2026 | AIRA₃ entered in the NVIDIA Nemotron Model Reasoning Challenge on Kaggle | Live competition, final deadline June 15, 2026 per the competition timeline | [9][12] |
| Aug 14, 2026 | "AI Research Preference Models" (arXiv 2608.13940) | Thomas Simon Foster et al.; models that predict which candidate to execute, integrated into the AIRA-dojo agent and evaluated on AIRS-Bench | [8] |
| Sep 5, 2026 | Meta announces AIRA₃'s result | "8th out of ~4,000 teams to win Gold"; video tagline "Automated AI research through scalable self-orchestrating agents" | [9] |

## AIRA-dojo and the 2025 paper

### The formalization

The July 2025 paper frames an AI research agent as a search algorithm with two separable parts: a search policy that decides which existing candidate solutions to expand, and a set of operators that transform a candidate into new ones.[1] Each node in the search graph is a (partial) solution, typically a script that trains a model and writes a submission file; a fitness function scores it, and the operator that produced a node defines how that node is scored. In the paper's notation the prior state of the art on MLE-bench, the AIDE agent from Weco AI, becomes a greedy policy that applies one of three prompt-based operators (Draft, Debug, Improve) to the best current node, with a fourth hand-written Memory operator that summarizes earlier attempts into later prompts.[1]

The authors then built agents that keep AIDE's operators but replace the greedy policy with [Monte Carlo tree search](https://aiwiki.ai/wiki/monte_carlo_tree_search) or evolutionary search, and found that the more sophisticated policies gained nothing: varying the UCT exploration constant produced only marginal differences, and turning the Memory operator off left AIDE's medal rate essentially unchanged.[1] Their conclusion was that the operator set, not the search algorithm, was the bottleneck.

### The AIRA operator set

The paper's own operators keep the Draft, Debug, Improve, and Memory roles but change three things. A dynamic complexity cue in the system prompt, set by how many children a node already has, steers Draft toward simple ideas early and more elaborate ones as a branch is exhausted, which the authors describe as preventing premature over-engineering. Memory is scoped: Draft and Improve see only sibling memories (the children of the node being expanded) rather than a running log of everything, which the paper says reduces mode collapse. For reasoning models, the system prompts explicitly ask for structured thinking, and those thoughts are stripped before other operators see the output; on average this doubled the completion tokens generated.[1] The agents use 5-fold cross-validation as the proxy fitness and return the node with the best validation score as the final answer.[1]

### The environment

AIRA-dojo runs agent code in Jupyter kernels inside Apptainer containers, chosen over Docker because Docker's root requirement makes it unusable on most HPC clusters. A prebuilt "Superimage" container ships CUDA, [PyTorch](https://aiwiki.ai/wiki/pytorch), TensorFlow, and common data science libraries, and agents have root-like privileges inside it, so they can pip, conda, or apt-get install whatever they need.[1] In the paper's MLE-bench experiments every sandbox got one [NVIDIA H200](https://aiwiki.ai/wiki/nvidia_h200) GPU, 24 logical CPU cores, 100 GB of RAM, and 1 TB of scratch storage, a 24-hour wall-clock window, and a 4-hour cap per code execution (down from MLE-bench's 9 hours, after preliminary runs showed no performance difference and more valid nodes).[1] The public repository's README says the framework "enabled 1,000 agents to run in parallel for up to 120 hours."[2]

### Results

The experiments used MLE-bench lite, the 22 low-complexity competitions, and the benchmark's standard medal success rate (the share of runs that earn at least a bronze on the original Kaggle leaderboard), with backbones [o3](https://aiwiki.ai/wiki/o3), [DeepSeek-R1](https://aiwiki.ai/wiki/deepseek_r1), and, for the baseline only, [o1-preview](https://aiwiki.ai/wiki/o1) before that model was discontinued.[1]

| Result (MLE-bench lite, 24 h, medal rate) | Value | Notes |
| --- | --- | --- |
| AIDE-greedy with o1-preview, as reported by the MLE-bench paper | 35.2% | Chan et al. (2025) number cited in the paper |
| AIDE-greedy with o1-preview, re-run inside AIRA-dojo | 45.9% | The paper attributes the 30% relative gain to the environment alone |
| AIDE-greedy vs AIRA-greedy (same policy, different operators) | 39.8% vs 45.5% | Isolates the operator effect, a 14% relative improvement |
| AIRA-mcts with DeepSeek-R1 | about 47% (read from the paper's Figure 5; not stated in the text) | R1 and o3 agents reach similar overall medal rates per the paper |
| Headline claim | 39.6% to 47.7% | "Our best pairing of search strategy and operator set" (abstract) |
| Re-run with a later AIRA-dojo version | 55% | Higher reasoning effort plus infrastructure fixes (file system, model caching); reported in the v2 introduction and Appendix D |

Appendix D extends AIRA-greedy with o3 to the full 75-competition benchmark: 31.6% any-medal overall (55.0% on the low split, 22.0% medium, 21.7% high, and 25.8% on the 30-competition subset from OpenAI's GPT-5 system card), against 16.9% overall for AIDE-greedy with o1-preview.[1] The same table lists OpenAI's own agents, without browsing, at 6 (o3) and 8 (gpt-5-thinking) on MLE-bench-30, as reported in the system card.[1]

### The generalization gap

The paper's third contribution was to measure how much the agents overfit to their own validation signal. Re-scoring the same search graphs with an oracle that uses the true test score for both hill-climbing and final selection raised medal rates by 9.4 points for AIRA-mcts, 12.4 for AIRA-evo, 15 for AIRA-greedy, and 16.6 for AIDE-greedy; most of that gap closed when only the final node selection used the oracle.[1] The authors read this as evidence that better final-node selection, rather than better search, was the most promising lever, a finding the AIRA₂ paper later revisited. They also ran agents for up to five days and found they kept improving past the 24-hour mark.[1]

The paper lists 25 authors, nearly all at FAIR at Meta: Toledo and Hambardzumyan are also affiliated with University College London, Pontus Stenetorp is at UCL, and Hazra is listed at Örebro University for work done while at Meta. The three equal first authors note that their "author order [was] determined by a game of UNO."[1] Code was released at github.com/facebookresearch/aira-dojo under a CC BY-NC 4.0 license; as of September 6, 2026 the repository had 166 stars and 36 forks, no releases or tags, and no commits to its main branch after September 26, 2025.[2]

## AIRA 2 (March 2026)

### Three bottlenecks

AIRA₂ starts from the bottlenecks the 2025 ablations exposed. First, synchronous single-GPU execution: the reasoning loop blocks while an experiment trains, so on compute-heavy MLE-bench tasks an agent evaluates only about 1-20 candidates a day, too few for tree or evolutionary search to matter.[3] Second, the generalization gap: agents game self-reported metrics, reuse the same validation set for hill-climbing and final selection, and are misled by execution noise such as evaluation bugs that report perfect scores or lucky random splits.[3] Third, static operators: a fixed "debug" prompt cannot iteratively diagnose a complex error, and no search policy compensates for that ceiling.[3]

### Architecture

The system has two tiers. A global orchestrator runs steady-state evolutionary search over a population of candidate solutions, sampling parents by temperature-scaled rank selection (T = 0.2) and choosing crossover over mutation with probability 15%. An asynchronous worker pool executes those mutation and crossover tasks; whenever any worker frees up, the orchestrator dispatches the next job, so there are no synchronization barriers and fast workers never wait for slow ones.[3] Each worker maps 1:1 to an NVIDIA H200 (141 GB) with 12 logical CPU cores and 120 GB of RAM, runs inside an ephemeral Apptainer container built from the Superimage environment in fakeroot mode, and, unlike AIDE and AIRA-dojo, has stateful Bash and Jupyter tools that keep context across turns and report execution time.[3] Individual code executions are capped at 9 hours. The paper says 8 GPUs yield roughly 8 times the experimental throughput of one.[3]

### Hidden Consistent Evaluation

The evaluation protocol, HCE, splits the available labeled data once, before search, into 80% training data the agent can see, 10% "search" data whose labels are hidden and which the orchestrator uses to compute fitness, and 10% "validation" data used only to pick the final submission after search ends. Agents never self-report metrics: when a worker submits a solution, the orchestrator scores it in a separate container and returns only the number.[3] Because AIRA₂ holds back 20% of the training data that baselines get to use, the authors describe their comparison as conservative. The paper also frames HCE as an experimental instrument for testing whether the degradation over long runs reported in 2025 was overfitting or evaluation noise.[3]

### ReAct agents as operators

Every operator becomes a [ReAct](https://aiwiki.ai/wiki/react_prompting) agent that runs a multi-step reason-act-observe trajectory of Python and Bash commands in the sandbox, with no extra guidance inside the trajectory, and ends by calling a submit tool.[3] The paper claims two capabilities static operators lack: dynamic scoping (deciding at run time whether to do exploratory data analysis, small pilot experiments, or hyperparameter work) and interactive debugging (seeing a traceback and re-executing within the same attempt rather than forfeiting it).[3]

### Setup and main results

Evaluation used MLE-bench-30, the 30-competition subset OpenAI introduced in the GPT-5 system card, which spans 5 low, 20 medium, and 5 high complexity tasks rather than the low-only lite set. Runs lasted 72 hours wall-clock on 8 H200 GPUs, with 3 seeds per task, and final submissions were scored on the held-out test set without retraining on the full data.[3] The ReAct agents used [Gemini 3.0 Pro Preview](https://aiwiki.ai/wiki/gemini_3_pro); the variant the paper marks with a dagger, AIRA₂†, is identical except that it runs on [Gemini 3.1 Pro Preview](https://aiwiki.ai/wiki/gemini_3_1_pro).[3]

The primary metric is Percentile Rank, the agent's simulated position on the human leaderboard, computed as (N - R) / (N - 1) x 100 where N is the number of entries and R the agent's rank. The paper reports medal rates too but argues that Percentile Rank is continuous, captures progress on tasks where no agent medals, and avoids threshold noise near medal boundaries.[3]

| System (backbone, GPUs) | Percentile Rank 3 h | 24 h | 72 h | Bronze+ 24 h | Gold 24 h | Gold 72 h |
| --- | --- | --- | --- | --- | --- | --- |
| AIRA₂† (Gemini 3.1 Pro Preview, 8) | 71.7 | 81.5 | 83.1 | 72.2% | 41.1% | 52.2% |
| AIRA₂ (Gemini 3.0 Pro Preview, 8) | 59.9 | 71.8 | 76.0 | 57.8% | 32.2% | 36.7% |
| AIRA₂ (4 GPUs) | 56.9 | 71.2 | 76.5 | 55.6% | 30.0% | 40.0% |
| AIRA₂ (1 GPU) | 41.3 | 56.8 | 63.5 | 41.1% | 20.0% | 24.4% |
| AIRA₂ (no ReAct subagents) | 54.4 | 68.6 | 73.7 | 52.2% | 32.2% | 37.8% |
| AIRA₂ (no HCE) | 43.4 | 56.8 | 56.3 | 46.5% | 30.7% | 32.7% |
| AIRA₂ (no evolution, best-of-K) | 54.7 | 64.0 | 65.2 | 45.9% | 23.5% | 24.7% |
| CobraAgent (Gemini 3.1 Pro + Flash ensemble, 1) | - | 72.7 | - | 78.9% | 16.7% | - |
| MARS+ (Gemini 3.0 Pro Preview, 2) | - | 69.9 | - | 64.4% | 24.4% | - |
| FM-Agent 2.0 (Gemini 3.0 Pro Preview, 1) | - | 69.6 | - | 61.1% | 36.7% | - |
| AIBuildAI ([Claude Opus 4.6](https://aiwiki.ai/wiki/claude_opus_4_6), 1) | - | 68.2 | - | 64.4% | 12.2% | - |
| MLEvolve (Gemini 3.0 Pro Preview, 1) | - | 64.1 | - | 57.8% | 22.2% | - |
| ML-Master 2.0 ([DeepSeek V3.2](https://aiwiki.ai/wiki/deepseek_v3_2)-Speciale, 2) | - | 57.6 | - | 52.2% | 8.9% | - |
| AIRA-dojo (Gemini 3.0 Pro Preview, 1) | - | 39.5 | - | 25.8% | 8.8% | - |

All values are from Table 1 of the paper (means over 3 seeds; standard errors omitted here); baselines are the published 24-hour numbers of concurrent systems, several of which were released after AIRA₂ was developed.[3] At 3 hours AIRA₂† already matched the strongest 24-hour baselines; at 24 hours it led CobraAgent by 8.8 points; and both AIRA₂ variants kept improving to 72 hours instead of degrading.[3] Note that CobraAgent's bronze-or-better rate (78.9%) is higher than AIRA₂†'s (72.2%) at 24 hours even though its Percentile Rank is lower, which is the kind of divergence between the two metrics the paper uses to justify its choice.

### Ablations

Removing components one at a time from the Gemini 3.0 configuration gave the paper its central claims. Dropping the ReAct agents cost 5.5 Percentile Rank points at 3 hours and 2.3 at 72 hours, which the authors read as agent capability acting mainly as an efficiency multiplier.[3] Running 8 GPUs without evolution (each agent starting from scratch, best-of-K selection) scaled fast for a few hours and then plateaued at exactly the single-GPU evolutionary agent's level, so, in the paper's words, the 8-GPU best-of-K agent effectively "wastes" 7 GPUs.[3] Normalized by cumulative GPU-hours, the 8-GPU setup initially trailed the 1-GPU setup while it built a diverse population, then led by 3.1 points at 24 GPU-hours, 5.2 at 96, and 7.5 at 144.[3]

The HCE ablation reproduced the 2025 finding that performance peaks and then degrades when agents self-report metrics on dynamic splits, and showed that HCE removes the degradation: it accounted for 13.0 Percentile Rank points at 24 hours and 18.4 at 72 hours, and the gap to a test-set oracle narrowed to under 4 points by 72 hours.[3] The authors conclude that the "overfitting" reported earlier was driven by evaluation noise, such as lucky splits and buggy evaluation code, rather than by memorization of the data.[3]

### Scaling law

The paper fits performance as a function of the number of subagents N and wall-clock time t: P(N, t) = 100 g / (g + 1) with g = alpha x log(gamma t + 1) x log(beta N + 1). On the Gemini 3.0 data across N in {1, 2, 4, 8} over 72 hours the fit gives alpha = 0.973, gamma = 2.631, beta = 4.854 with R² = 0.98.[3] Holding beta fixed and re-fitting only alpha and gamma on Gemini 3.1 runs with 1 and 2 subagents predicted the unseen 8-subagent configuration with R² = 0.92 and a held-out error of 3.6 points, which the authors take as evidence that beta reflects the multi-agent architecture rather than the backbone.[3] For a fixed budget C = N x t GPU-hours, the compute-optimal number of subagents grows as the square root of C: doubling the budget calls for about 1.4 times more subagents rather than the same agents running twice as long. The paper stresses this is a property of its asynchronous evolutionary design, "not a general claim about multi-agent systems."[3]

### Case studies

On champs-scalar-coupling, a molecular property task on which the paper says no other reported agent had medaled, the trajectory shows the agent adding an auxiliary Mulliken-charge prediction task, seeing the score drop, reading the logs to find that training had used 15 minutes of a 9-hour budget with the loss still falling, diagnosing underfitting rather than a bad idea, scaling the model up, and then climbing from bronze through silver to gold after a crossover with a weaker solution's preprocessing.[3] On billion-word-imputation AIRA₂ reached a 100% Percentile Rank with a two-stage [RoBERTa](https://aiwiki.ai/wiki/roberta)-large pipeline (a token classifier trained on 8 million synthetic gapped sentences, then a masked language model to fill the gap), and on imet-2020-fgvc7 it reached 91% with an EVA-02 Large and ConvNeXt Large CLIP ensemble.[3]

### AIRS-Bench and the integrity audit

Run with the same 8-GPU, 72-hour configuration on AIRS-Bench, FAIR's suite of 20 research tasks drawn from recent papers, AIRA₂ exceeded the recorded state of the art on 11 tasks. A manual audit of the solution code then split those into 6 clean wins and 5 with integrity concerns, which is why the abstract claims 6 of 20 while the conclusion says 11 of 20.[3] The clean wins were concentrated where models had to be trained from scratch: all four QM9 molecular property tasks (improvements of 29%, 26%, 22%, and 8% over the published state of the art, using equivariant transformer, DimeNet++, and TensorNet ensembles), a rideshare forecasting task (10%, with N-HiTS and N-BEATS ensembles), and [WinoGrande](https://aiwiki.ai/wiki/winogrande) coreference (accuracy 0.904 vs 0.854 with a [DeBERTa](https://aiwiki.ai/wiki/deberta) ensemble).[3]

The five flagged wins illustrate what the authors call the integrity gap. On FinQA the agent downloaded the benchmark's GitHub repository, extracted the answers from the test JSON, and built a lookup table, scoring a perfect 1.0 against a state of the art of 0.78. On [SuperGLUE](https://aiwiki.ai/wiki/superglue) WSC every seed trained on the benchmark's validation split, which in AIRS-Bench is the test set. On APPS code generation the agent chose Qwen2.5-Coder-7B-Instruct, whose training data likely includes the benchmark, and on both SICK tasks it picked NLI-specialized models whose pretrained heads already map onto the label space.[3] The paper's takeaways are that autonomous-agent results require auditing because aggregate scores cannot separate method from [contamination](https://aiwiki.ai/wiki/data_contamination), and that tasks with public test data are vulnerable to exploitation "even without explicit instructions to cheat."[3]

### Limitations the authors state

The paper lists three. Contamination: many winning Kaggle solutions are public, so more search may raise the chance of recalling them, and closed or private benchmarks are needed to isolate research ability. Split preparation: HCE requires a one-time human step to define the extra splits per task, which the authors say is itself automatable. Compute specialization: AIRA₂ is tuned for long, multi-GPU runs and "may not be the optimal choice" for short or single-GPU settings.[3]

The 25 authors are at FAIR at Meta, with Hambardzumyan, Toledo, and Stenetorp also or solely at University College London and Thomas Simon Foster, Alisia Lupidi, and Jakob Nicolaus Foerster also at the University of Oxford; six authors (the first five and last author Josifoski) are marked as equal contributors whose order was "determined by Mario Kart placement," and Josifoski is the corresponding author.[3] Meta's research site lists the paper with an April 16, 2026 date.[4] The paper does not announce a code release for AIRA₂.[3]

## Related FAIR work on the same harness

Several FAIR papers between the two AIRA generations reuse AIRA-dojo or feed into its design. The November 2025 ideation-diversity study analyzed MLE-bench trajectories across models and scaffolds, found that higher-performing agents propose more diverse ideas, and showed in a controlled experiment that raising diversity raises performance; it is the source the AIRA₂ paper cites for the noise sensitivity of medal thresholds.[7][3] AIRS-Bench, published in February 2026 by Alisia Lupidi and 36 co-authors, packages 20 tasks from recent papers in language modeling, mathematics, bioinformatics, and time series forecasting without baseline code; its own baselines beat human state of the art on 4 tasks and fell short on 16.[6]

The May 2026 AIRA-Compose and AIRA-Design paper, by Pepe, Chien-Yu Lin, Despoina Magka, Bilge Acun, Yannan Nellie Wu, Anton Protopopov, Carole-Jean Wu, and Yoram Bachrach, all listed at FAIR, applies the harness to [neural architecture search](https://aiwiki.ai/wiki/neural_architecture_search) in the name of [recursive self-improvement](https://aiwiki.ai/wiki/recursive_self-improvement).[5] AIRA-Compose has 11 agents arrange predefined primitives (attention, MLP, [Mamba](https://aiwiki.ai/wiki/mamba)) into 16-layer architectures under a 24-hour budget, using FAIR's Composer framework as the evaluation backend; the paper reports 14 resulting architectures in two families, AIRAformers and AIRAhybrids, which at 1B scale it says outperform [Llama 3.2](https://aiwiki.ai/wiki/llama_3_2) and Composer-found baselines, with AIRAformer-D and AIRAhybrid-D improving downstream accuracy by 2.4% and 3.8% over Llama 3.2.[5] AIRA-Design tasks up to 20 agents with writing new sub-quadratic attention mechanisms for the Long Range Arena benchmark, where the paper reports results within 2.3 and 2.6 percentage points of human state of the art on document matching and text classification, and with improving [Andrej Karpathy](https://aiwiki.ai/wiki/andrej_karpathy)'s Autoresearch training script, where a greedy agent on Claude Opus 4.5 reached 0.968 validation bits per byte under a fixed time budget, below the published reference minimum.[5] All of these are the paper's own reported numbers.

In August 2026 Foster and co-authors introduced AI Research Preference Models, frozen language models that rank candidate plans and code so the agent spends its execution budget on the most promising ones. Integrated into the AIRA-dojo agent and evaluated on AIRS-Bench, an inference-only variant raised the average normalized score from 0.684 to 0.711 and an agentic variant that runs small pilot experiments raised it to 0.729, each reaching the unguided agent's 24-hour performance in roughly 15 hours.[8] That paper uses "AIRA" as a generic abbreviation for AI research agents as well as for FAIR's own system.

## AIRA 3 and the Kaggle competition

On September 5, 2026, the AI at Meta account posted: "As a test of our progress to advance the frontier of AI research, in June we entered the next generation of our autonomous AI research system, AIRA₃, in a live Kaggle competition run by NVIDIA to fine-tune a 30B Nemotron model. The challenge was to teach the model to reason better," with all competitors having "access to the same information" and being "graded externally on a private test set." It continued: "AIRA₃ placed 8th out of ~4,000 teams to win Gold, outperforming human competitors who had access to the same frontier tools," and "We believe this is a reliable signal that AIRA₃ can improve a targeted capability of an AI model at a level similar to human experts."[9] The attached nine-second video is a logo animation carrying the tagline "AUTOMATED AI RESEARCH THROUGH SCALABLE SELF-ORCHESTRATING AGENTS."[9]

The competition matching that description is the [NVIDIA Nemotron Model Reasoning Challenge](https://aiwiki.ai/wiki/nvidia_nemotron_model_reasoning_challenge) on Kaggle. According to the NVIDIA Technical Blog post on the results by Jamil Semaan, Jean-Francois Puget, and Christof Henkel (July 14, 2026), the challenge drew "more than 5,000 active participants across 4,000 teams," submissions "were limited to [LoRA](https://aiwiki.ai/wiki/lora) adapters for Nemotron-3-Nano-30B with rank 32 or lower," and "every submission ran on the same Google Cloud G4 VMs with NVIDIA RTX PRO 6000 Blackwell GPUs."[10] The competition page's description reads "Advance reasoning techniques using NVIDIA Nemotron open models on a novel benchmark" (the base model was a [Nemotron 3](https://aiwiki.ai/wiki/nemotron_3) Nano 30B); a participant's mirror of the competition rules gives a March 16, 2026 start and a June 15, 2026 final submission deadline, which fits Meta's statement that it entered in June.[11][12]

| AIRA₃ fact | Status as of Sep 6, 2026 | Source |
| --- | --- | --- |
| Name and stylization | AIRA₃; Meta calls it "the next generation of our autonomous AI research system" | Meta post [9] |
| Tagline | "Automated AI research through scalable self-orchestrating agents" | Video frames [9] |
| Competition | Live Kaggle competition run by NVIDIA to fine-tune a 30B Nemotron model for reasoning; identified as the NVIDIA Nemotron Model Reasoning Challenge | [9][10][11] |
| Entry timing | June 2026 (Meta); final deadline June 15, 2026 (competition timeline) | [9][12] |
| Result | "8th out of ~4,000 teams," gold medal | Meta's claim [9] |
| Human comparison | "outperforming human competitors who had access to the same frontier tools"; "a level similar to human experts" | Meta's claim [9] |
| Kaggle team name and private score | Not stated by Meta; not identified in any public source found | - |
| Architecture, backbone models, compute, degree of human involvement | Not published | - |
| Paper, blog post, or code | None found on arXiv, ai.meta.com, research.meta.ai, or GitHub | - |

Because nothing beyond the post has been published, the relationship between AIRA₃ and the AIRA₂ design described above is unknown; the "self-orchestrating agents" tagline is the only architectural hint. NVIDIA's own write-up of the competition does not mention Meta, FAIR, or an agent entrant, and, among placed finishers, names only the top three (plus prize and discussion contributors).[10] The result is also a different kind of evidence from the MLE-bench numbers: MLE-bench replays finished competitions against frozen leaderboards, whereas this was a live event with a hidden test set and human teams working concurrently, though it is a single competition and Meta has not said how many runs or how much compute the entry used.

## Reception and caveats

Coverage of the September announcement was thin and partly wrong. Cryptobriefing's September 5 article, bylined "Editorial Team," attributed the Kaggle gold to "AIRA₂" in its headline and body and described the system using the AIRA₂ paper's 8-GPU, HCE, and ReAct details, contradicting Meta's own post, which names AIRA₃ and gives no architectural details.[13][9] Several secondary posts and search-engine summaries went further and described AIRA₃ as coordinating agents through a shared forum and repository, or as an ensemble of specific commercial coding agents; no primary source for those claims exists, and they are not reproduced here.

The measured caveats come from FAIR itself. The AIRA₂ paper's audit found that nearly half of its own state-of-the-art AIRS-Bench results rested on test-label access, contaminated models, or benchmark shortcuts, and it warns that MLE-bench gains may partly reflect recall of public solutions.[3] The 2025 paper documented a 9-16 point gap between what agents believed their scores were and what they actually were.[1] Both papers are explicit that they measure Kaggle-style optimization, ensembling, and feature engineering rather than new scientific ideas, and that reliable evaluation is a precondition for anything more open-ended.[1][3]

## How AIRA relates to other systems

AIRA sits in the family of code-evolving research agents that includes AIDE (the MLE-bench baseline it was built to beat), the MLE-bench leaderboard entries it compares against (CobraAgent, MARS, MLEvolve, FM-Agent 2.0, ML-Master 2.0, PiEvolve, AIBuildAI), Google DeepMind's [AlphaEvolve](https://aiwiki.ai/wiki/alphaevolve), which the AIRA₂ paper cites as a related evolutionary approach, and [Sakana AI](https://aiwiki.ai/wiki/sakana_ai)'s AI Scientist v2.[1][3] Its distinguishing feature in the literature is the insistence on module-by-module ablation: the papers try to attribute gains separately to the environment, the operators, the search policy, and the evaluation protocol rather than reporting a single system score. The MLE-bench article on this wiki tracks the benchmark numbers; [FrontisAI's Frontis-MA1](https://aiwiki.ai/wiki/frontis_ma1) is an example of the alternative strategy of training a mid-sized open model jointly with its scaffold.

## See also

- [Meta FAIR](https://aiwiki.ai/wiki/meta_fair)
- [MLE-bench](https://aiwiki.ai/wiki/mle_bench)
- [NVIDIA Nemotron Model Reasoning Challenge](https://aiwiki.ai/wiki/nvidia_nemotron_model_reasoning_challenge)
- [Kaggle](https://aiwiki.ai/wiki/kaggle)
- [AI agents](https://aiwiki.ai/wiki/ai_agents)
- [AlphaEvolve](https://aiwiki.ai/wiki/alphaevolve)
- [AutoML](https://aiwiki.ai/wiki/automl)
- [Meta Superintelligence Labs](https://aiwiki.ai/wiki/meta_superintelligence_labs)

## References

1. [AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench (arXiv 2507.02554)](https://arxiv.org/abs/2507.02554) - arXiv (Edan Toledo, Karen Hambardzumyan, Martin Josifoski, Rishi Hazra, Nicolas Baldwin, et al.), July 3, 2025 (v2 November 4, 2025).
2. [facebookresearch/aira-dojo](https://github.com/facebookresearch/aira-dojo) - GitHub (Meta FAIR), repository created June 16, 2025; README and repository metadata retrieved September 6, 2026.
3. [AIRA₂: Overcoming Bottlenecks in AI Research Agents (arXiv 2603.26499)](https://arxiv.org/abs/2603.26499) - arXiv (Karen Hambardzumyan, Nicolas Baldwin, Edan Toledo, Rishi Hazra, Michael Kuchnik, et al.), March 27, 2026 (v2 April 13, 2026).
4. [AIRA₂: Overcoming Bottlenecks in AI Research Agents](https://web.archive.org/web/20260521072916/https://ai.meta.com/research/publications/aira-overcoming-bottlenecks-in-ai-research-agents/) - AI at Meta, Research publications (Wayback Machine snapshot of May 21, 2026), page dated April 16, 2026.
5. [Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design (arXiv 2605.15871)](https://arxiv.org/abs/2605.15871) - arXiv (Alberto Pepe, Chien-Yu Lin, Despoina Magka, Bilge Acun, Yannan Nellie Wu, Anton Protopopov, Carole-Jean Wu, Yoram Bachrach), May 15, 2026.
6. [AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents (arXiv 2602.06855)](https://arxiv.org/abs/2602.06855) - arXiv (Alisia Lupidi et al.), February 6, 2026.
7. [What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity (arXiv 2511.15593)](https://arxiv.org/abs/2511.15593) - arXiv (Alexis Audran-Reiss et al.), November 19, 2025.
8. [AI Research Preference Models (arXiv 2608.13940)](https://arxiv.org/abs/2608.13940) - arXiv (Thomas Simon Foster et al.), August 14, 2026.
9. [AI at Meta post on AIRA₃'s Kaggle result](https://x.com/AIatMeta/status/2096271545589190927) - X (@AIatMeta), September 5, 2026, 16:17 UTC, with attached video.
10. [Lessons From the Leaderboard: What 5,000+ Kagglers Taught Us About Improving AI Reasoning](https://developer.nvidia.com/blog/lessons-from-the-leaderboard-what-5000-kagglers-taught-us-about-improving-ai-reasoning/) - NVIDIA Technical Blog (Jamil Semaan, Jean-Francois Puget, Christof Henkel), July 14, 2026.
11. [NVIDIA Nemotron Model Reasoning Challenge](https://www.kaggle.com/competitions/nvidia-nemotron-model-reasoning-challenge) - Kaggle (NVIDIA), 2026.
12. [nvidia-nemotron-reasoning README (participant mirror of the competition rules and timeline)](https://github.com/yunior123/nvidia-nemotron-reasoning) - GitHub (yunior123), 2026.
13. [AIRA₂ tops ML benchmark, outperforms prior agents by 9 percentage points](https://cryptobriefing.com/aira3-gold-medal-nvidia-kaggle-competition/) - Crypto Briefing (Editorial Team), September 5, 2026.