NVIDIA Nemotron Model Reasoning Challenge

The NVIDIA Nemotron Model Reasoning Challenge was a Kaggle competition run by NVIDIA from March to June 2026 in which participants tried to improve the reasoning accuracy of a fixed open model, Nemotron 3 Nano 30B-A3B, on a new benchmark of logical reasoning puzzles. Entrants could not submit a full model: every submission was a LoRA adapter of rank 32 or lower, evaluated with fixed inference code, without internet access, on identical Google Cloud G4 virtual machines equipped with NVIDIA RTX PRO 6000 Blackwell GPUs.^[1] Kaggle's description of the competition was "Advance reasoning techniques using NVIDIA Nemotron open models on a novel benchmark."^[2]

NVIDIA reported that more than 5,000 active participants across 4,000 teams took part, producing thousands of submissions and over 1,000 discussion posts.^[1] The competition drew attention outside the Kaggle community in September 2026, when Meta said its autonomous research system AIRA₃ (see AIRA) had been entered as a live competitor in June and had placed 8th of roughly 4,000 teams, which Meta described as a gold-medal finish.^[14] NVIDIA published a retrospective on July 14, 2026, "Lessons From the Leaderboard: What 5,000+ Kagglers Taught Us About Improving AI Reasoning," written by Jamil Semaan, Jean-Francois Puget and Christof Henkel, which names the top three solutions and distills five lessons from the leaderboard and forum; NVIDIA's developer forum carries a mirror of the post with the same date.^[1][17]

Origins and launch

The competition was announced during NVIDIA's GTC 2026 conference. In a March 16, 2026 Google Cloud blog post, Mark Lohmeyer, Google Cloud's VP and GM for AI and Computing Infrastructure, wrote that "to enable the community to get hands-on with NVIDIA Nemotron on Google Cloud, we are also launching the NVIDIA Nemotron model reasoning challenge on Kaggle, powered by G4 VMs," and described the goal as improving "Nemotron 3 Nano's reasoning accuracy on a new benchmark using techniques such as prompting, synthetic data generation, data curation, and fine-tuning."^[3] The same post announced a preview of fractional G4 VMs and expanded Vertex AI Model Garden support for the Nemotron 3 family, which places the competition inside a broader Google Cloud and NVIDIA partnership push around G4 instances.^[3]

NVIDIA's developer community team hosted a virtual kickoff event on Luma, listed as starting at 00:00 UTC on March 17, 2026 (the evening of March 16 in California) and hosted by "Jamil | NVIDIA"; the event page records 803 attendees. Its description said the challenge would be "evaluated using Nemotron models and a new reasoning benchmark developed by the NVIDIA research team," and that participants would be "free to explore your own approaches: prompting, data filtering, synthetic data generation, reinforcement learning, and lightweight fine-tuning."^[4] NVIDIA framed the competition as a contribution to an open ecosystem "where reasoning methods can be studied, reproduced, and extended by the broader AI community," pointing to the Nemotron family's open weights, datasets and training recipes.^[4]

A participant's public repository, created March 24, 2026, mirrors the Kaggle competition page and gives the timeline below. Because Kaggle competition pages are rendered client-side and could not be read directly for this article, dates from that mirror are labeled as such; the start date is consistent with the Luma kickoff and the Google Cloud announcement, and the June 15 close is consistent with Meta's statement that it entered in June.^{[3][4][5][14]}

Scroll sideways for more →

The task

The benchmark consisted of logical reasoning puzzles in which the solver has to identify a hidden transformation rule from examples and apply it to a new case. The participant mirror of the competition description lists "bit manipulation, algebraic equations, etc." as example domains, and NVIDIA's retrospective adds cryptarithms, describing the model's job as having "to infer the hidden transformation, produce any reasoning trace, and return the final answer within the token budget."^[1][5] The training file train.csv contained an id, a prompt and an answer for each puzzle; test.csv contained ids and prompts, with a hidden test set substituted at scoring time.^[5] One participant's writeup states that the training set held 9,500 puzzles, all of which the author eventually solved with deterministic code, and that this alone was still not enough for a gold medal.^[7]

Scoring was accuracy on the hidden set. According to the participant mirror, the model had to place its final answer inside a LaTeX-style \boxed{} wrapper, and a prediction counted as correct if it matched the ground truth as an exact string or fell within a tolerance of 0.01 for numeric answers.^[5] A public notebook titled "NVIDIA Nemotron Metric," published under Kaggle's competition-metrics account, is listed by the participant mirror as the competition metric, and a "NVIDIA Nemotron Submission Demo" notebook by Ryan Holbrook showed entrants how to package an adapter.^[5][21][22] Scores reported in participant writeups are fractions of puzzles solved, for example a private-leaderboard score of 0.86 for two different silver-medal entries.^[6][7]

The base model was published on Kaggle under the same competition-metrics account as "Nemotron-3-Nano-30B-A3B-BF16."^[12] It corresponds to NVIDIA's Hugging Face release NVIDIA-Nemotron-3-Nano-30B-A3B-BF16, a hybrid Mamba-2 and Transformer mixture-of-experts model with roughly 31.6 billion total parameters (per the safetensors metadata) and about 3 billion active per token, released in December 2025 under the NVIDIA Nemotron Open Model License.^[13] The competition data was released under a CC BY 4.0 license according to the participant mirror.^[5]

Rules and constraints

NVIDIA's retrospective summarizes the constraints that shaped the competition: "Participants couldn't use internet access at evaluation time, modify the inference code, or submit a full model. Submissions were limited to LoRA adapters for Nemotron-3-Nano-30B with rank 32 or lower, and final scoring happened on a private leaderboard."^[1] Every submission ran on the same cloud hardware, which NVIDIA said let "teams focus on reasoning workflows instead of infrastructure management while working within realistic constraints on throughput, memory and cost that mirror how these systems run in production."^[1]

Scroll sideways for more →

The fixed inference configuration, as recorded by the participant mirror, is reproduced below. These values were not confirmed on the Kaggle page itself; the 8,192-token context and 7,680-token generation cap are consistent with the repeated emphasis on the "token budget" in NVIDIA's retrospective and in participant writeups.^[1][5]

Scroll sideways for more →

Greedy decoding at temperature 0 did not make scoring fully deterministic: NVIDIA's retrospective cites a forum thread by a participant named Taha on non-determinism, warning that "when repeated submissions can move by a few points, validation needs to measure stability, not just peak score."^[1]

Prizes

The participant mirror lists a total prize pool of $106,388, combining cash with NVIDIA DGX Spark desktop systems. The Open Progress Prize was tied to the April 9 midpoint cut-off, and the mirror marks the three Open Contribution Awards as open only to teams finishing in the top 10 percent.^[5] Tong Hui Kang's blog confirms that the Open Progress Prize carried prize money.^[10] None of the following figures could be confirmed on the Kaggle page directly.

Scroll sideways for more →

Participation and results

NVIDIA's count of "more than 5,000 active participants across 4,000 teams" is the organizer's figure.^[1] A silver-medal writeup gives its author's final standing as "Rank 179/4354," which suggests the private leaderboard listed 4,354 teams once it was finalized; Meta rounded the field to "~4,000 teams."^[6][14] The 4,354 figure comes from a single participant and could not be checked against the leaderboard, which Kaggle renders only in the browser.

The final leaderboard itself was not accessible for this article, so placements and names below are taken from NVIDIA's retrospective and from the titles of public Kaggle writeups. NVIDIA names the top three as "Team re's 1st-place solution," "the 2nd-place writeup, from vli," and "YS-L's 3rd-place writeup."^[1] Kaggle hosts writeups titled "1st Place Solution," "2nd place solution" and "3rd Place Solution" for the competition; the second is summarized by its author as "a solution that reverse-engineers the data-generation process and improves bit manipulation from 85% to 99%."^[8] A further writeup, "2nd public / 6th private place solution" by Alehandreus and Yurnero, records a team that led the public leaderboard's second position but finished sixth on the private set, an illustration of the public-private gap that NVIDIA's fifth lesson is about.^[1][9]

The Open Progress Prize went to Tong Hui Kang, who wrote on May 2, 2026 that he had won it and that it was "the first time I won prize money from Kaggle competitions."^[10] His public notebook is titled "End-to-end finetuning for LB 0.85," and his repository, which by September 2026 had accumulated more than 100 stars, describes a pipeline of reasoning-trace generation, augmentation, corpus construction and LoRA supervised fine-tuning, together with a static site for inspecting per-problem base-model behavior, synthetic-rule investigation status, token-level training corpus statistics and per-run training metrics.^[11] NVIDIA credits his bit-manipulation representation as "a foundation for later solutions" because it "avoided wasteful brute-force reasoning while keeping useful structure inside the model's completion budget."^[1] A later writeup, "LB15: Extension to @huikang's progress prize solution," shows how directly other teams built on it.^[19]

Other participants named in NVIDIA's retrospective include Shehab Anwer (an "ATLAS" discussion on verified traces and augmented solvers), Mayur Pawar ("Breaking the SFT Ceiling," on solver engineering and executable chain-of-thought audits), a participant posting as StSTXion (training on the constraint-satisfaction search process for cryptarithms), EnDream (per-category error analysis) and Yurnero (validation methodology).^[1] Two silver-medal writeups both report a private score of 0.86: "How Far Can Verified Synthetic Data Go?" (deterministic solvers, LoRA placement and model soups) and "I Solved Every Single Puzzle - And It Still Wasn't Enough," whose author cracked all 9,500 training puzzles deterministically with a "data-first approach" and still finished in silver.^[6][7] A bronze-medal writeup, "Progressive Reasoning Fine-Tuning," describes moving "From Answer-Only SFT to Distilled Long-Form CoT."^[20]

Whether Open Contribution Awards were given, and to whom, is not stated in any source consulted for this article.

Meta's AIRA3 entry

On September 5, 2026, the AI at Meta account on X wrote that "in June we entered the next generation of our autonomous AI research system, AIRA₃, in a live Kaggle competition run by NVIDIA to fine-tune a 30B Nemotron model," that "all competitors had access to the same information and were graded externally on a private test set," and that "AIRA₃ placed 8th out of ~4,000 teams to win Gold, outperforming human competitors who had access to the same frontier tools." Meta called the result "a reliable signal that AIRA₃ can improve a targeted capability of an AI model at a level similar to human experts."^[14] The post named neither the competition nor the team account used on Kaggle; the identification with this challenge follows from the base model, the timing and the field size, all of which match.^[1][14]

The AIRA line is a family of AI agents for machine learning research developed at Meta FAIR with academic collaborators; earlier versions were evaluated offline on MLE-bench, which replays past Kaggle competitions against their historical leaderboards. The Nemotron entry differs in that the agent competed live against current participants for the same hidden test set. As of September 6, 2026, Meta had not published a paper, blog post or Kaggle writeup describing AIRA₃'s methods, and the placement rests on Meta's statement alone; a September 5 Cryptobriefing article that repeated the 8th-place result labeled the system "AIRA₂" in its headline and body, contradicting Meta's own naming.^[14][18] Which Kaggle team name corresponds to the Meta entry could not be confirmed.

Kaggle's medal thresholds depend on the number of teams. The MLE-bench paper reproduces Kaggle's 2024 progression rules as a table in which competitions with 1,000 or more teams award gold to the "Top 10 + 0.2%*" of teams, silver to the top 5 percent and bronze to the top 10 percent, with the footnote that "the threshold increases by 1 for every 500 additional teams."^[15] An 8th-place finish falls inside the gold band under those rules; the exact gold cutoff for this competition is not stated in the sources consulted.

Five lessons from the leaderboard

NVIDIA's retrospective organizes what worked into five lessons, each illustrated with named solutions, and refers readers to a recap stream for "what NVIDIA Kaggle Grandmasters observed across the competition."^[1]

Scroll sideways for more →

Verifiable traces

The first lesson is that synthetic data helped only when the reasoning traces were checked. NVIDIA writes that "a reasoning trace can look convincing while still teaching the wrong shortcut," and recommends treating traces "like code or math proofs: each step should be checkable." The winning solution "generated synthetic problems, attached solver-generated traces, and used SFT to train the model on those traces"; the second-place writeup "described a similar workflow, with separate files for generating synthetic prompts and the reasoning traces the model trained on."^[1]

Token budget as part of the problem

Because generation was capped, "long traces could contain the right logic but still fail if the model ran out of room, repeated too much scaffolding, or spent too many tokens representing simple data." The top three solutions extended Tong Hui Kang's bit-manipulation representation "with HEX, hybrid hex-binary signatures, and compacted Hui Kang-style traces," which NVIDIA presents as the clearest example of representation choices changing what a small model can do inside a fixed context.^[1]

Memory versus computation

NVIDIA's third lesson distinguishes stable structure (schemas, formulas, operator patterns, symbolic mappings) from the part of each problem that changes. Team re "used a signature catalog for cryptarithm patterns, letting the model rely on reusable structure before doing a shorter consistency check"; the retrospective stresses that "the point was not to memorize answers" but to avoid spending reasoning steps on structure that could be precomputed.^[1]

Tools upstream, not at inference

Since no external program could run at evaluation time, tools were used to manufacture and audit training data. Mayur Pawar's writeup "used solver engineering, executable chain-of-thought audits, and failure-driven synthetic data to find cases where answer-correct traces were not teaching a valid solving process," and StSTXion's approach "trained on the search process itself: candidate choices, constraint propagation, contradictions, backtracking, and commits."^[1]

Measuring by task type

The last lesson is about validation under a hidden test set: "a model may get better at symbolic search, worse at arithmetic, and unchanged on retrieval-heavy tasks, while the average barely moves." EnDream's breakdown "separated formatting success from real reasoning quality," and the Alehandreus and Yurnero team used "full-training validation and per-domain public checks to understand which changes helped which task types."^[1][9]

Techniques in context

The competition was, in effect, a controlled experiment in post-training a small open reasoning model under a parameter budget: the adapter rank cap limited how much could be learned, the fixed token budget limited how long the model could think, and the ban on inference-time tools moved all of the engineering into data. The organizers' retrospective concludes that "improving reasoning performance is not about one magic prompt, one bigger dataset, or one training trick," and lists five habits: verifiable traces, budget-aware reasoning, specialized solvers for structured tasks, validation against real failure modes, and training choices "that preserve reasoning behavior, not just leaderboard score."^[1]

Participants used ordinary open tooling. The competition description, as mirrored, allowed "any framework (HuggingFace, Unsloth, Axolotl, TRL, etc.)," and public writeups reference LoRA placement choices and model soups (Unsloth, Axolotl and TRL are the frameworks named).^[5][6] Tong Hui Kang's repository runs its pipeline as a sequence of Python scripts (reasoning, augmentation, corpus, train_sft.py) and uploads the adapter from a cloud job.^[11] Although reinforcement learning was an allowed technique and had its own Open Contribution Award, the solutions NVIDIA singles out are supervised fine-tuning on solver-verified traces; the retrospective does not describe any prize-winning RL method.^[1][5]

NVIDIA also used the competition as a showcase for its G4 cloud instances. The retrospective's closing section says the challenge "ran on Google Cloud G4 VMs with NVIDIA RTX PRO 6000 Blackwell GPUs, giving participants access to the performance and memory needed to fine-tune, run inference, iterate on prompts and data pipelines, and evaluate Nemotron models against real benchmarks," and invites developers to reproduce the setup on the same instances.^[1] Google Cloud's GTC post positioned G4 as suited to fine-tuning and inference of models "ranging from 30B to more than 100B parameters."^[3]

Follow-up events

NVIDIA's developer YouTube channel carries a Nemotron Labs session titled "What 5,000 Kagglers Taught Us About Improving AI Reasoning," whose page metadata gives a publication date of June 30, 2026 and whose description invites viewers to "join NVIDIA Kaggle Grandmasters and challenge winners for a live discussion on the techniques that moved the leaderboard."^[16] The July 14 retrospective links to a replay of "the Nemotron Labs recap stream" and announces a further live discussion with the winning teams on July 24, 2026.^[1] Whether that second session took place as planned was not confirmed for this article.

The competition sits alongside NVIDIA's other competitive-benchmark activity with the Nemotron family in 2026, including the AI gold-medal result at the 2026 IOI obtained with a Nemotron 3 derivative. For this competition, NVIDIA's in-house Kaggle Grandmasters were the authors of the retrospective and hosts of the recap sessions.^[1][16]

See also

• Nemotron 3
• AIRA (Meta AI Research Agents)
• Kaggle
• MLE-bench
• LoRA (Low-Rank Adaptation)
• Nemotron 3.5 Lightning
• Nemotron-Labs-TwoTower

References

1. ^Lessons From the Leaderboard: What 5,000+ Kagglers Taught Us About Improving AI Reasoning - NVIDIA Technical Blog (Jamil Semaan, Jean-Francois Puget, Christof Henkel), July 14, 2026.
2. ^NVIDIA Nemotron Model Reasoning Challenge - Kaggle competition page (competition id 129716), accessed September 6, 2026.
3. ^Google Cloud and NVIDIA expand AI innovation across industries at GTC 2026 - Google Cloud Blog (Mark Lohmeyer), March 16, 2026.
4. ^NVIDIA Nemotron Model Reasoning Challenge - Luma event page (NVIDIA Developer Community Team), event of March 17, 2026 UTC, accessed September 6, 2026.
5. ^yunior123/nvidia-nemotron-reasoning README - GitHub (participant mirror of the competition page: timeline, prizes, evaluation rules, inference parameters), created March 24, 2026.
6. ^How Far Can Verified Synthetic Data Go? - Kaggle participant writeup (summary: "Private 0.86, Rank 179/4354, Deterministic solvers, LoRA placement, and model soups"), 2026.
7. ^I Solved Every Single Puzzle - And It Still Wasn't Enough - Kaggle participant writeup (summary: "9,500/9,500 puzzles cracked deterministically. Private 0.86, Silver medal. Data-first approach."), 2026.
8. ^2nd place solution - Kaggle writeup (summary: "A solution that reverse-engineers the data-generation process and improves bit manipulation from 85% to 99%"), 2026.
9. ^2nd public / 6th private place solution - Kaggle writeup by Alehandreus and Yurnero, 2026.
10. ^Winning the Nemotron Progress Prize - Huikang's blog (Tong Hui Kang), May 2, 2026.
11. ^tonghuikang/nemotron: Progress Prize winning submission for NVIDIA Nemotron Model Reasoning Challenge - GitHub, repository created March 24, 2026, accessed September 6, 2026.
12. ^Nemotron-3-Nano-30B-A3B-BF16 - Kaggle Models (Kaggle Competition Metrics account), accessed September 6, 2026.
13. ^nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 - Hugging Face model card and API metadata (created December 4, 2025; NVIDIA Nemotron Open Model License), accessed September 6, 2026.
14. ^AI at Meta post on AIRA₃'s Kaggle result - X (@AIatMeta), September 5, 2026.
15. ^MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering - arXiv (Chan et al., OpenAI), Table 1 reproducing Kaggle's 2024 medal thresholds, October 2024.
16. ^What 5,000 Kagglers Taught Us About Improving AI Reasoning | Nemotron Labs - YouTube (NVIDIA Developer), page metadata dated June 30, 2026.
17. ^Lessons From the Leaderboard: What 5,000+ Kagglers Taught Us About Improving AI Reasoning - NVIDIA Developer Forums mirror of the blog post, July 14, 2026.
18. ^AIRA₂ tops ML benchmark, outperforms prior agents by 9 percentage points - Cryptobriefing (Editorial Team), September 5, 2026.
19. ^LB15: Extension to @huikang's progress prize solution - Kaggle participant writeup, 2026.
20. ^Bronze Medal Solution: Progressive Reasoning Fine-Tuning - Kaggle participant writeup (summary: "From Answer-Only SFT to Distilled Long-Form CoT"), 2026.
21. ^NVIDIA Nemotron Submission Demo - Kaggle notebook (Ryan Holbrook), 2026.
22. ^NVIDIA Nemotron Metric - Kaggle notebook (Kaggle Competition Metrics account), 2026.