AI Habitat
Last reviewed
Jun 3, 2026
Sources
13 citations
Review status
Source-backed
Revision
v1 · 1,608 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Jun 3, 2026
Sources
13 citations
Review status
Source-backed
Revision
v1 · 1,608 words
Add missing citations, update stale details, or suggest a clearer explanation.
AI Habitat (usually just Habitat) is an open-source simulation platform for embodied AI research, developed primarily by Meta AI (the group then known as Facebook AI Research, or FAIR) together with academic collaborators. It lets researchers train and evaluate virtual robots and agents on tasks such as navigation, instruction following, and object rearrangement inside photorealistic 3D reconstructions of real indoor spaces, with the goal of developing skills in simulation before transferring them to physical robotics [1][2]. The platform is built around two main pieces: a fast 3D simulator called Habitat-Sim and a higher-level Python library called Habitat-Lab (originally named Habitat-API) for defining tasks, configuring agents, and running training and benchmarking [1][8][9].
Habitat is deliberately split so that the heavy graphics-and-physics work is separated from the research code that defines experiments.
Habitat-Sim is the low-level engine: a flexible, high-performance 3D simulator with configurable agents, configurable sensors, and generic handling of 3D datasets [1]. It loads scanned indoor and outdoor environments, CAD models, and object assets, renders sensor observations (RGB, depth, and semantic segmentation), and, in later versions, simulates rigid-body dynamics through the Bullet physics engine [8]. Its main design goal is speed. The original 2019 paper reports that when rendering a scene from Matterport3D, Habitat-Sim achieves several thousand frames per second running single-threaded and can exceed 10,000 frames per second multi-process on a single GPU [1]. Running the simulator far faster than real time is what makes large-scale reinforcement learning, which can require billions of agent steps, practical.
Habitat-Lab is the modular high-level library for end-to-end development of embodied AI algorithms: it defines tasks (for example navigation, instruction following, and question answering), configures agents and sensors, and handles training and benchmarking [1][9]. It uses Habitat-Sim as its underlying simulator [9]. A companion component, Habitat-Baselines, ships reference reinforcement learning implementations (such as PPO-based training) so that researchers have working baselines to compare against [9]. The library was renamed from Habitat-API to Habitat-Lab in release v0.1.6 to better reflect that it is a toolkit for running experiments rather than a thin API [10].
Habitat supports a range of embodied tasks, which have grown with each major release [1][2][3]:
Alongside the platform, Meta has run the Habitat Challenge, an annual embodied-navigation competition hosted on the EvalAI platform. Rather than submitting predictions, participants upload code (as Docker containers) that is run against held-out scenes on cloud GPUs, so that agents are judged on how well they generalize to unseen environments [4][7]. The first edition, Habitat Challenge 2019, was organized by FAIR's A-STAR group and focused on the PointGoal task with separate RGB and RGB-D input tracks; it opened on April 2, 2019 [7]. Later editions shifted to harder tasks: the 2023 challenge ran ObjectNav and ImageNav on HM3D-Semantics scenes [11].
Habitat has gone through three numbered generations, each described in its own peer-reviewed paper.
| Version | Year / venue | Key additions |
|---|---|---|
| Habitat 1.0 | 2019 (ICCV 2019) | Original Habitat-Sim and Habitat-API; fast photorealistic rendering of static 3D scans; PointGoal navigation and the first Habitat Challenge [1][7] |
| Habitat 2.0 | 2021 (NeurIPS 2021) | Interactive, physics-enabled scenes; the ReplicaCAD apartment dataset with articulated furniture; the Home Assistant Benchmark for rearrangement and mobile manipulation [2] |
| Habitat 3.0 | 2023 preprint, published at ICLR 2024 | Simulated humanoid avatars; human-in-the-loop control via keyboard/mouse or VR; Social Navigation and Social Rearrangement collaboration tasks [3][6] |
Habitat 1.0 (2019). The platform was introduced in "Habitat: A Platform for Embodied AI Research" by Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Erik Wijmans, and colleagues at ICCV 2019 [1]. It emphasized rendering speed and a clean separation between the simulator and the research library, and it used static (non-interactive) 3D scans. The paper reported cross-dataset generalization experiments across Matterport3D and Gibson with several sensor configurations (blind, RGB, depth, and RGB-D) [1].
Habitat 2.0 (2021). "Habitat 2.0: Training Home Assistants to Rearrange their Habitat" by Andrew Szot, Alexander Clegg, Eric Undersander, and colleagues added interactivity and physics [2]. It contributed three things: ReplicaCAD, an artist-authored, reconfigurable 3D dataset of apartments with articulated objects such as cabinets and drawers that open and close; the Home Assistant Benchmark (HAB), a suite of household tasks (tidy the house, prepare groceries, and set the table) that test mobile manipulation; and a faster physics-enabled simulator reported to exceed 25,000 simulation steps per second (about 850 times real time) on an 8-GPU node, roughly a 100-fold speedup over prior work [2]. The paper also found that flat reinforcement-learning policies struggled on HAB relative to hierarchical approaches, that hierarchies of independent skills suffered "hand-off" problems between skills, and that classical sense-plan-act pipelines were more brittle than learned policies [2].
Habitat 3.0 (2023 / ICLR 2024). "Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots" by Xavier Puig, Eric Undersander, Andrew Szot, and colleagues added simulated humans so that robots can be trained to collaborate with people [3][6]. Avatars are driven by SMPL-X, a data-driven parametric body model, and the paper notes the humanoid simulation runs roughly an order of magnitude faster than existing humanoid simulators while still handling diverse appearance and motion [5]. A human-in-the-loop tool lets a real person control an avatar with mouse and keyboard or through a virtual-reality interface, so policies can be evaluated against actual humans as well as scripted ones [5]. The work studied two collaborative tasks, Social Navigation (the robot finds and follows a human while keeping a safe distance) and Social Rearrangement (a robot and a human jointly rearrange objects), and observed emergent cooperative behavior such as the robot yielding space when it blocks a human's path [3][5].
Habitat is designed to work with many external 3D scene datasets rather than shipping a single fixed world. The 3D scans provide the environments agents move through, while separate object datasets (such as YCB and Google Scanned Objects) supply manipulable items [8]. The most commonly used scene datasets are below.
| Dataset | What it provides |
|---|---|
| Matterport3D (MP3D) | Building-scale RGB-D scans of real indoor spaces with semantic annotations, used in the original Habitat experiments [1] |
| Gibson | Photorealistic scans of indoor environments, used for cross-dataset navigation experiments [1] |
| Replica | High-fidelity reconstructions of indoor spaces with dense semantic labels [8] |
| ReplicaCAD | Artist-authored, reconfigurable apartments with articulated, interactive objects, introduced for Habitat 2.0 [2] |
| HM3D (Habitat-Matterport 3D) | 1,000 large-scale building reconstructions, the largest such dataset for academic research at release [8][13] |
HM3D was released by Meta and Matterport in 2021 as the Habitat-Matterport 3D Dataset and presented at the NeurIPS 2021 Datasets and Benchmarks track [13]. It contains 1,000 building-scale 3D reconstructions of real interiors, including multi-floor homes and stores, with about 112,500 square meters of navigable space, which the authors report as 1.4 to 3.7 times larger than building-scale datasets such as MP3D and Gibson, and with fewer reconstruction artifacts and higher visual fidelity [13]. A semantic-annotation extension, HM3D-Semantics, adds object labels and is used in recent Habitat Challenge editions [8][11].
Habitat-Sim and Habitat-Lab remain open source on GitHub under Meta's facebookresearch organization, and the platform has been widely used in embodied-AI navigation and manipulation research. As of the v0.3.4 release, Meta's internal teams noted that the project is no longer receiving active development or maintenance from them, though the code and datasets remain publicly available [9].