NetHack Learning Environment
The NetHack Learning Environment (NLE) is an open-source reinforcement learning environment built around NetHack, a single-player, turn-based roguelike game first released in 1987.[1] It was introduced by Heinrich Küttler, Nantas Nardelli, Alexander H. Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette and Tim Rocktäschel, working at Facebook AI Research (Meta FAIR), the University of Oxford, New York University, Imperial College London and University College London, in a paper posted to arXiv on 24 June 2020 and accepted at NeurIPS 2020.[1] The authors presented NLE as an environment that is both fast to simulate and deep enough to drive long-term research on exploration, planning, skill acquisition and language-conditioned reinforcement learning, and they argued that existing testbeds rarely offered both properties at once.[1] NLE later served as the platform for the NeurIPS 2021 NetHack Challenge, for the NetHack Learning Dataset of human and bot games, and for the NetHack component of the BALROG benchmark for language model agents.[6][7][13] As of November 2026, the best reported agents, neural, symbolic and LLM-based, remained far from winning the full game.[6][9][14][16]
Background: why NetHack
NetHack is a descendant of Hack, itself an open-source take on the original Rogue game. The player controls a hero who must descend through more than 50 procedurally generated dungeon levels, retrieve the Amulet of Yendor, escape upward and pass five final levels (the four Elemental Planes and the Astral Plane) to offer the amulet to a deity, which is called ascending.[1] The layout of each level, the position of landmarks such as the Oracle and the structure of branches such as the Gnomish Mines are randomized, so a player is very unlikely to see the same situation twice.[1]
The NLE paper lists several properties that make the game attractive for research:[1]
- Procedural generation and stochasticity. Because states rarely repeat, the authors argued that exploration methods which depend on returning to previously visited states, such as Go-Explore, are unsuitable, and that agents must generalize instead of memorizing action sequences.
- Rich, compositional dynamics. The paper cites the NetHack Wiki's count of more than 450 items and 580 monster types. Its worked example of crossing water to reach Medusa's Island involves grease, towels, snow boots, a kraken and Medusa's petrifying gaze.
- Very long episodes. Successful expert games usually last tens of thousands of turns, and average successful runs can last hundreds of thousands. The paper states that this makes NetHack episodes one or two orders of magnitude longer than those of StarCraft or Dota 2.
- External knowledge and data. Human players rely on the NetHack Wiki and community "spoilers", which the authors suggested could support language-assisted learning, and the nethack.alt.org (NAO) servers hold recordings of more than five million human games.
- Low compute cost. The game is terminal based, so it runs far faster than 3D environments.
The paper also discussed earlier roguelike environments. It credited gym_nethack, a Gym environment based on NetHack 3.6.0, as the first to propose NetHack for reinforcement learning, but noted that it removed boulders, traps, locked doors and item identification to simplify the game. NLE was designed instead to keep the game unchanged and as close as possible to what human players experience.[1]
Design
Interface
NLE uses NetHack 3.6.6, released on 8 March 2020, as an unmodified backend. The authors added control over the random number generator for seeding and code that exposes the game's internal state to a Python front end, and they wrapped the result in an OpenAI Gym interface.[1] The current version is based on NetHack 3.6.7 and uses the Farama Foundation's Gymnasium API.[2]
By default, an observation contains several named arrays:[1]
| Observation key | Content |
|---|---|
glyphs, chars, colors, specials | The 2D symbolic view of the dungeon map |
blstats | "Bottom-line stats" such as coordinates, hit points, strength, dexterity and hunger |
message | The message currently shown at the top of the screen |
inv_glyphs, inv_strs, inv_letters, inv_oclasses | Padded arrays describing the hero's inventory |
The full action space has 93 actions, covering every action a human player can take: 16 movement actions (eight one-step compass directions and eight "move far" directions) and 77 commands such as eating, opening, kicking, reading and praying.[1] NLE also shipped with a dashboard for replaying and analyzing recorded games.[1] A community-built NLE Language Wrapper translates the observations into text and accepts text actions, which makes the environment usable by large language models.[2][13]
Speed
The authors compared environments through their Python Gym interfaces; all except Obstacle Tower were run for 60 seconds with standard settings. NLE ran close to Procgen and much faster than Atari or 3D environments:[1]
| Environment (Gym interface) | Environment steps per second |
|---|---|
| CartPole-v1 | 76,880 |
| NLE (score task) | 14,400 |
| Procgen (CoinRun) | 13,130 |
| Gym Retro (Airstriker-Genesis) | 1,310 |
| Arcade Learning Environment (Montezuma's Revenge) | 900 |
| Obstacle Tower | 60 |
| MineRL (NavigateDense) | 60 |
Tasks
The original release defined a suite of tasks for tractable subgoals, and the authors noted that new tasks can be written in a few lines of code.[1]
| Task | Goal |
|---|---|
| Staircase | Reach the staircase down to the next level |
| Pet | Reach the staircase while accompanied by the hero's pet |
| Eat | Find and eat edible items |
| Gold | Collect gold |
| Score | Maximize the in-game score |
| Scout | Discover unseen parts of the dungeon |
| Oracle | Find the Oracle |
In June 2021 NLE version 0.7.0 added a "challenge" task (registered as NetHackChallenge-v0) that exposes the full game: 113 actions covering the full keyboard, the raw terminal output among the observations, and a random race, role, alignment and gender for every episode.[5][6][7]
Evaluation protocol
The paper defined a goal for the environment: NLE counts as solved once agents can be trained to ascend ten times in a row, without retries, on unseen seeds with a randomly chosen character.[1] Because the authors judged that goal to be out of reach for the foreseeable future, they recommended comparing methods on the in-game score in the meantime, while noting its weaknesses: expert players can win while deliberately keeping their score low. They asked researchers to report the full character specification, the game options and allowed actions, any hard-coded behaviors and the number of training seeds, and to report the average score over 1,000 episodes on unseen seeds. Saving and reloading during an episode ("save scumming") and manipulating the random number generator are forbidden at test time.[1]
Baseline results
The NeurIPS 2020 paper trained a recurrent policy with IMPALA, a distributed actor-critic method, as implemented in TorchBeast, for one billion environment steps. The model embeds each of NetHack's 5,991 glyphs, encodes the full map and a 9 by 9 crop around the agent with convolutional networks, encodes the stats with an MLP and feeds the result to an LSTM.[1] The authors also tested Random Network Distillation (RND), an exploration bonus, and trained four characters: a monk, a valkyrie, a wizard and a tourist.[1]
- On the staircase task, success rates ranged from 50.42% for the tourist to 80.42% for the wizard, and the agents learned to kick open locked doors and search for hidden passages. RND raised success rates for three of the four roles and lowered it for the wizard.[1]
- On the score task, average scores were 748 for the monk, 573 for the valkyrie, 314 for the wizard and 11 for the tourist. The best monk episode reached dungeon level 11 with a score of 4,260.[1]
- No agent found the Oracle, apart from one valkyrie episode the authors described as lucky.[1]
- When trained on a limited set of seeds, agents generalized poorly to held-out seeds; the gap between training and test performance narrowed once agents trained on at least 1,000 seeds.[1]
NeurIPS 2021 NetHack Challenge
The NeurIPS 2021 NetHack Challenge asked participants to build an agent, either trained or hand-coded, that could ascend in NLE.[6] It was hosted on the AIcrowd platform and sponsored by Meta AI and DeepMind.[6] The organizers' report, with Eric Hambro and Sharada Mohanty as first authors, was posted to arXiv in March 2022 and published in the proceedings of the NeurIPS 2021 Competitions and Demonstrations Track (PMLR volume 176).[6]
| Item | Detail |
|---|---|
| Environment | NLE challenge task, frozen at version 0.7.3 for evaluation |
| Duration | 144 days, development phase June to October 2021, test phase in the last two weeks of October |
| Participation | 483 registered entrants, 631 submissions |
| Prize pool | $20,000 across four tracks |
| Tracks | Best Overall Agent, Best Neural Agent, Best Symbolic Agent, Best Agent from an Academic Lab or Independent Researcher |
| Ranking | Number of ascensions, then median in-game score, then mean in-game score |
| Final evaluation | 4,096 episodes per submission |
All of these details come from the organizers' report.[6] No agent ascended. The organizers reported that symbolic bots beat deep reinforcement learning by a wide margin: the top three places in the overall track went to symbolic agents, and the best symbolic bot's median score was nearly three times that of the best neural entry.[6] The winning agent set a new NLE record with a median score of 5,300, which the report noted is only just above what the NetHack Wiki calls a beginner's score. About 1 in 20 of its Valkyrie games scored more than 30,000.[6] Neural agents still improved on the 2020 baseline's median score by nearly five times.[6]
The report summarizes the leading approaches in the teams' own words:[6]
- AutoAscend (symbolic, first place). A hand-written bot that parses the screen into a model of the game state and selects strategies through a custom behavior tree. Strategies such as exploration, combat and emergency healing can interrupt one another. The NetHack Learning Dataset paper later described it as the best open-source artificial agent for NetHack 3.6.6, having won "by a considerable margin".[6][7]
- RAPH (hybrid, best neural-track entry). A rule-based high-level controller in the style of the options framework that hands control to hard-coded skills, with a neural network trained by reinforcement learning for combat.[6]
- Kakao Brain (neural). An IMPALA-based agent trained end to end, with a separated action space, a GRU and a transformer over usable items and spells, and separate role-specific agents for the Healer, Ranger, Rogue, Tourist and Wizard roles. Each policy trained for 10 billion steps on 16 V100 GPUs.[6]
The organizers explained the gap partly by the ease with which symbolic bots encode strategy-like subroutines such as "Find Minetown", while hierarchical reinforcement learning, long-term credit assignment and the use of human prior knowledge remain open problems for neural agents.[6] A team taking part in the challenge also released a PPO baseline built on the Sample Factory framework, which reached a mean score of roughly 700, against about 400 for the official TorchBeast baseline, in under 24 hours on one RTX 2080 Ti.[6]
NetHack Learning Dataset
In November 2022 Hambro and colleagues released the NetHack Learning Dataset (NLD), presented in the NeurIPS 2022 Datasets and Benchmarks track, for imitation learning and offline reinforcement learning on NetHack.[7] Its loading code now ships with NLE.[2]
| Part | Source | Episodes | Transitions | Compressed size |
|---|---|---|---|---|
| NLD-NAO | Human games on nethack.alt.org, 2009 to 2020 (states only) | 1,511,228 | 9,858,127,896 | 229 GB |
| NLD-AA | AutoAscend playing NetHackChallenge-v0 (states, actions and scores) | 109,545 | 3,481,605,009 | 96.7 GB |
The dataset loader, TtyrecDataset, streams compressed terminal recordings. The authors reported reading all of NLD-NAO in under 15 hours on 10 CPUs.[7] In their experiments, offline methods fell short of the performance in the data by a wide margin. Behavioral cloning did better than offline RL but still well below the bot that generated the data, and methods that combined offline data with online training worked best.[7]
Later research
NLE became a common testbed for research on long-horizon decision making. Selected work:
| Work | Year | Contribution |
|---|---|---|
| MiniHack (Samvelyan et al.) | 2021 | A sandbox for designing custom RL environments from NetHack's entities and dynamics, published in the NeurIPS 2021 Datasets and Benchmarks track[8] |
| NetHack is Hard to Hack (Piterbarg, Pinto, Fergus) | 2023 | Extended AutoAscend to record its strategy choices, producing a large demonstration dataset. The resulting neural agent beat earlier fully neural policies but did not close the gap to symbolic bots (NeurIPS 2023)[9] |
| Motif (Klissarov et al.) | 2023 | Built an intrinsic reward from an LLM's preferences between pairs of game-message captions. An agent trained only on that reward reached a higher game score than one trained directly on the score[10] |
| NetPlay (Jeurissen et al.) | 2024 | Described as the first LLM-powered zero-shot agent for NetHack, choosing among predefined skills[12] |
| BALROG (Paglieri et al.) | 2024 | Included NLE as the hardest of six game environments for LLM and VLM agents (ICLR 2025)[13] |
| MaestroMotif (Klissarov et al.) | 2024 | Used LLM feedback to design a reward for each skill and LLM-written code to combine the trained skills, evaluated on NLE tasks[11] |
| Scalable Option Learning (Henaff et al.) | 2025 | A hierarchical RL algorithm trained on 30 billion frames of NetHack, which the authors report surpasses flat agents[15] |
| Revisiting the NetHack Learning Environment (Matthews et al.) | 2026 | An ICLR 2026 blog-track post arguing that constraints on NLE's observation and action spaces leave much of NetHack's complexity inaccessible to agents, and proposing interface changes that the authors report improve performance[16] |
Piterbarg and colleagues put the challenge gap differently from the organizers: symbolic agents beat neural ones "by over four times in median game score". Their neural agent improved median score over earlier fully neural policies by 127% offline and 25% online, but they concluded that "mere scaling is insufficient" to match the best symbolic bots or top human players.[9]
Use as an LLM agent benchmark
In BALROG, models receive a text rendering of the NetHack state from the language wrapper, plus an ASCII map. In the vision-language mode the ASCII map is replaced by an image of the game's tile map.[13] Because the in-game score tracks real progress poorly, the BALROG authors built a progression metric from the human games in NLD. It is based on the probability that a human player who reached a given dungeon level or experience level went on to win, scaled from 0% at the start of the game to 100% at ascension.[13]
In the original BALROG evaluation, every model barely moved on NetHack. OpenAI's o1-preview, tested only on this environment because of its cost, scored highest with an average progression of 1.57%, and the best single run (by Gemini 1.5 Pro in vision-language mode) reached dungeon level 3 and experience level 4.[13] The authors described a "knowing-doing" gap: models could answer questions correctly about why leaving the dungeon on the first level or eating rotten food is fatal, yet often did both in play.[13] They also noted that one demonstration of NetHack play can take more than 700,000 input tokens to represent, which makes in-context learning from examples very expensive.[13]
Scores have risen since then but remain low. On the BALROG language-model leaderboard as of November 2026, the highest NetHack progression was 13.2% for a GPT-6 Astra configuration (entry dated 2026-09-18), an entry not marked as checked by the BALROG team; the highest entry marked as checked was Gemini 3 Pro at 6.8%.[14] The leaderboard explains that a check means the BALROG team received access to the system and reproduced its results.[14]
Development and maintenance
| Date (PyPI upload) | Release or event |
|---|---|
| 2020-05-05 | First PyPI release, version 0.1.0[5] |
| 2020-06-24 | NLE paper posted to arXiv (NeurIPS 2020)[1] |
| 2021-06-08 | Version 0.7.0 adds the challenge task[5][6] |
| 2021-07-21 | Version 0.7.3, the version used for challenge evaluation[5][6] |
| 2022-11-24 | Version 0.9.0[5] |
| 2024-07-19 | Version 1.0.0 moves from OpenAI Gym to Gymnasium and to NetHack 3.6.7[4] |
| 2024-11-07 | Version 1.1.0 supports Gymnasium 1.0 and Python 3.12[4] |
| 2025-05-14 | Version 1.2.0 adds seeding parameters for deterministic levels[4][5] |
| 2026-04-25 | Version 1.3.0 adds pixel observations[4][5] |
NLE was first developed in the facebookresearch/nle repository on GitHub. That repository is now archived, and its README sends users to a new home at heiner/nle, a repository that now redirects to NetHack-LE/nle, where the project is maintained.[3][2] The package ships with a TorchBeast agent as a starting point[2] and is distributed on PyPI under the NetHack General Public License.[5]
Continue reading
- BALROGBALROG (Benchmarking Agentic LLM and VLM Reasoning On Games) is a benchmark suite for evaluating the agentic capabilities of large language models and vision language models inside long-horizonMentioned 8 times above
- Gym (OpenAI Gym / Gymnasium)Gym, often written as OpenAI Gym, is an open source Python toolkit for developing and comparing reinforcement learning algorithms, originally released by openai on April 27, 2016.Mentioned 7 times above
- Arcade Learning EnvironmentThe Arcade Learning Environment (ALE) is an open-source software framework that turns Atari 2600 video games into reinforcement learning environments, and the name of the benchmark built on it.Listed in See also
- Imitation LearningImitation learning is a family of methods for learning sequential behavior from demonstrations.Listed in See also
- Factorio Learning EnvironmentThe Factorio Learning Environment (FLE) is an open source benchmark and research framework that uses the industrial automation game Factorio to evaluate the long-horizon agentic capabilities of large language…Listed in See also
- Symbolic AISymbolic AI is the approach to artificial intelligence that models intelligence as the rule-governed manipulation of human-readable symbols, formal logic, and explicit knowledge representations rather than…Listed in See also
More to read
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22Heinrich Küttler, Nantas Nardelli, Alexander H. Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, Tim Rocktäschel. "The NetHack Learning Environment". NeurIPS 2020 (arXiv:2006.13760), 2020. arxiv.org/...2006.13760
- ^1 ^2 ^3 ^4 ^5NetHack-LE. "The NetHack Learning Environment" (README). GitHub, accessed November 2026. github.com/...nle
- ^Facebook Research. "nle" (archived repository README). GitHub, accessed November 2026. github.com/...nle
- ^1 ^2 ^3 ^4NetHack-LE. "Releases: NetHack-LE/nle". GitHub, accessed November 2026. github.com/...releases
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Python Package Index. "nle". PyPI, accessed November 2026. pypi.org/...nle
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18Eric Hambro, Sharada Mohanty, Dmitrii Babaev, et al. "Insights From the NeurIPS 2021 NetHack Challenge". Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, PMLR 176:41-52 (arXiv:2203.11889), 2022. arxiv.org/...2203.11889
- ^1 ^2 ^3 ^4 ^5 ^6Eric Hambro, Roberta Raileanu, Danielle Rothermel, Vegard Mella, Tim Rocktäschel, Heinrich Küttler, Naila Murray. "Dungeons and Data: A Large-Scale NetHack Dataset". NeurIPS 2022 Datasets and Benchmarks Track (arXiv:2211.00539), 2022. arxiv.org/...2211.00539
- ^Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel. "MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research". NeurIPS 2021 Datasets and Benchmarks Track (arXiv:2109.13202), 2021. arxiv.org/...2109.13202
- ^1 ^2 ^3Ulyana Piterbarg, Lerrel Pinto, Rob Fergus. "NetHack is Hard to Hack". NeurIPS 2023 (arXiv:2305.19240), 2023. arxiv.org/...2305.19240
- ^Martin Klissarov, Pierluca D'Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, Mikael Henaff. "Motif: Intrinsic Motivation from Artificial Intelligence Feedback". arXiv:2310.00166, 2023. arxiv.org/...2310.00166
- ^Martin Klissarov, Mikael Henaff, Roberta Raileanu, et al. "MaestroMotif: Skill Design from Artificial Intelligence Feedback". arXiv:2412.08542, 2024. arxiv.org/...2412.08542
- ^Dominik Jeurissen, Diego Perez-Liebana, Jeremy Gow, Duygu Cakmak, James Kwan. "Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents". arXiv:2403.00690, 2024. arxiv.org/...2403.00690
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Davide Paglieri, Bartłomiej Cupiał, Samuel Coward, et al. "BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games". ICLR 2025 (arXiv:2411.13543), 2024. arxiv.org/...2411.13543
- ^1 ^2 ^3BALROG team. "BALROG: Benchmarking Agentic LLM/VLM Reasoning On Games" (leaderboard). balrogai.com, accessed November 2026. balrogai.com
- ^Mikael Henaff, Scott Fujimoto, Michael Matthews, Michael Rabbat. "Scalable Option Learning in High-Throughput Environments". arXiv:2509.00338, 2025. arxiv.org/...2509.00338
- ^1 ^2Michael Matthews, Pierluca D'Oro, Anssi Kanervisto, Scott Fujimoto, Jakob Foerster, Mikael Henaff. "Revisiting the NetHack Learning Environment". ICLR 2026 Blog Track, 2026. iclr.cc/...10012114
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
2 revisions · v2 · 3,244 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check before publication on November 18, 2026: 112 claims checked against 19 cited and independent sources; 0 material and 5 minor corrections applied before publication.
Cite this page: AI Wiki. "NetHack Learning Environment." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/nethack_learning_environment
