Citation and evidence

robosuite

16 min full readUpdated 24 references

This article's verification

Report a problem with this article

Edit article
More

Use this article

Raw MarkdownExplore connections

Improve this page

Revision historyDiscuss this article

Topics

AI BenchmarksEmbodied AIOpen Source AIRobotics

Cite this article

robosuite is an open-source simulation framework and benchmark for robot learning, built on the MuJoCo physics engine. It provides a modular Python API for assembling robot manipulation tasks from robot models, objects and workspaces, a set of standardized benchmark environments, a library of robot controllers, and tools for collecting human demonstrations by teleoperation [1][2]. Researchers at the Stanford Vision and Learning Lab began it in late 2017 as an internal tool, and it is now maintained and used by that lab, the Robot Perception and Learning Lab at the University of Texas at Austin, and NVIDIA's Generalist Embodied Agent Research (GEAR) group [2]. The project belongs to the ARISE Initiative (Advancing Robot Intelligence through Simulated Environments), whose stated aim is to lower the barriers to entry for research at the intersection of AI and robotics [2]. The first major version, v1.0, was released in September 2020, and the current release line is v1.5 [2][3][6].

Several later simulation benchmarks and data-generation systems are built on or use robosuite, including robomimic, MimicGen, LIBERO and RoboCasa [12][14][15][16]. It is released under the MIT License [19].

History

Origins in SURREAL

The robosuite white paper says development of the framework started from the SURREAL project on distributed reinforcement learning for robot manipulation [1]. SURREAL was published in the proceedings of the 2nd Conference on Robot Learning (CoRL) in 2018 by Linxi Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese and Fei-Fei Li [5]. Alongside an open-source distributed RL framework, that paper introduced the SURREAL Robotics Suite, a set of simulated benchmark tasks for reproducible manipulation research [5]. The robosuite repository describes the code as originally developed in late 2017 in the Stanford Vision and Learning Lab (SVL) as an internal research tool [2], and the white paper dates the framework's debut to 2017 [1]. The earliest robosuite package on the Python Package Index (PyPI) was uploaded in October 2018 [6].

Version 1.0 and the white paper

robosuite v1.0, described in its release notes as "the first major version of robosuite", was published on GitHub on 2020-09-28 [3]. The accompanying white paper, "robosuite: A Modular Simulation Framework and Benchmark for Robot Learning", was posted to arXiv on 2020-09-25 by Yuke Zhu, Josiah Wong, Ajay Mandlekar and Roberto Martín-Martín, and described the v1.0 release [7]. A third version of the paper, dated 2025-01-18, was revised to cover v1.5 and added Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany and Yifeng Zhu as core members; it lists Zhu, Wong, Mandlekar and Martín-Martín as the founding members who initiated and lead the project [1][8]. The project's own citation entry points to the arXiv preprint, and the arXiv record lists no journal reference [2][8].

Release history

VersionDateMain changes
v1.02020-09-28First major version, with the white paper [3][7]
v1.12020-12-17Refactored infrastructure and standardized model classes for easier environment prototyping [2]
v1.22021-02-17Observable sensor models and dynamics randomization [2]
v1.32021-10-19New renderers (NVISII and iGibson), a standard renderer interface, and depth and segmentation modalities [9]
v1.42022-11-15Backend moved to DeepMind's official MuJoCo Python bindings; robot textures and bug fixes [2][10]
v1.52024-10-28Diverse robot embodiments including humanoids, custom robot composition, composite and whole-body controllers, more teleoperation devices, and photorealistic rendering [2][11]
v1.5.12025-02-08Patch release (PyPI upload date) [6]
v1.5.22025-12-24Patch release; changes include multiple-camera support, an xArm7 robot model and DualSense controller support [6][24]

Expanded article table

Dates for v1.1 to v1.5 are those given in the project's "Latest Updates" list [2].

The v1.3 release added two optional renderers next to MuJoCo's own. Its release notes describe NVISII as a high-fidelity ray-tracing renderer originally developed by NVIDIA, running at up to about 0.5 frames per second on a GTX 1080Ti GPU, and iGibson as a much faster renderer with physically based rendering, running at up to about 1,500 frames per second on the same GPU [9]. The same release added RGB, depth and segmentation observations (segmentation by class, instance or geom) across all three renderers [9].

The v1.4 release replaced the community mujoco-py bindings with DeepMind's official mujoco Python package. Its release notes say installation "has become much simpler", since MuJoCo can now be installed with pip install mujoco on Linux or Mac, and that because the new bindings do not ship with an onscreen renderer, the release added an OpenCV-based viewer [10].

The v1.5 release notes list five main features: support for complex robots including humanoids, custom robots built from modular components, a composite controller abstraction that includes whole-body controllers, more teleoperation devices (drag and drop in the MuJoCo viewer and Apple Vision Pro), and integration of NVIDIA Isaac Sim for real-time photorealistic rendering [11]. The release also turned on a lite_physics flag by default, which skips redundant simulation steps to speed up simulation [11].

"robosuite: A Modular Simulation Framework and Benchmark for Robot Learning", official video from Yuke Zhu, published September 28, 2020. Watch on YouTube.

Design

robosuite exposes two groups of APIs. Modeling APIs describe a simulation environment and task in code, and Simulation APIs wrap the physics engine and give external actors (a learned policy or a human using an input device) an interface for sending actions and receiving observations and rewards [1].

Modeling APIs

A simulation model is defined by a Task object that combines one or more robot models, zero or more object models, and a single Arena [1]. The robot model loads the robot together with its gripper models and base model from XML files. Objects can be loaded from 3D assets or generated procedurally by composing primitive shapes, and the Arena defines the workspace, such as a tabletop or bins, along with camera positions [1]. The Task merges these pieces into one XML model in MuJoCo's MJCF format, which MuJoCo then turns into a runtime simulation [1]. Each environment also takes a placement initializer, which samples valid, non-colliding starting positions for the objects at the start of every episode [1].

Simulation APIs

Environments follow the interface style of OpenAI Gym. An environment is created by calling make with the task name and options that set, among other things, on-screen or off-screen rendering, whether observations include object states or camera images, the control frequency, the episode horizon, and whether rewards are shaped or sparse [1]. The robots, grippers and controllers used in an environment are also arguments, so the same task can be run with different hardware and action spaces [1].

Robots, grippers and bases

The v1.5 paper lists models of 10 commercially available robots, 9 grippers (including a model of the Inspire dexterous hand), 4 bases (including an Omron wheeled mobile base) and 6 body-part controllers, with model properties taken from official product documentation or spec sheets [1]. A separately installed extension package, robosuite_models, adds 8 more robots, 8 grippers and 3 bases, and shows how to compose a custom robot from parts [1][13].

RobotMaker (as given in the paper)TypeDefault gripper or attachment in robosuite
PandaFranka Emika7-DoF armPandaGripper (parallel jaw)
SawyerRethink Robotics7-DoF armRethinkGripper (parallel jaw)
IIWAKUKA7-DoF armRobotiq 140 mm gripper
JacoKinova Robotics7-DoF armThree-finger gripper
Kinova3Kinova7-DoF armRobotiq 85 mm gripper
UR5eUniversal Robots6-DoF armRobotiq 85 mm gripper
BaxterRethink Robotics (now CoThink Robotics)Bimanual, two 7-DoF armsRethinkGripper on each arm
GR1Fourier Intelligence44-DoF humanoidDexterous hand on each arm
SpotBoston Dynamics12-DoF quadruped6-DoF arm
TIAGoPAL RoboticsBimanual mobile manipulatorTwo 7-DoF arms on a mobile base

Expanded article table

Source: robosuite v1.5 white paper [1].

Robots are designed to be plug-and-play: any combination of robot, gripper, base and controller can be used if the environment supports that configuration, and several copies of the same robot can be placed in one scene [1]. The paper recommends the Panda with operational space control for standardized comparisons, and uses it and the Sawyer for its benchmark experiments [1].

Controllers

Controllers turn high-level actions into joint torques for MuJoCo's simulated motors [1]. From v1.5, robosuite uses composite controllers: a composite controller takes one action vector and splits it into commands for body-part controllers for the arms, torso, head, base or legs [1]. Two composite controllers ship with the framework. BASIC passes each slice of the action to its body-part controller, so that, for example, the arms can use operational space control while a mobile base uses joint velocity control. WHOLE_BODY_IK treats the action as end-effector pose targets, solves inverse kinematics for joint angles, and passes those to joint position controllers [1]. Users can write their own by subclassing the CompositeController class, and the project includes an example third-party controller, WHOLE_BODY_MINK_IK, built on Kevin Zakka's Mink inverse kinematics library [1].

Body-part controllerControlsAction size (without gripper)
OSC_POSEEnd-effector position and orientation (operational space control)6; 12 with variable stiffness; 18 with variable impedance
OSC_POSITIONEnd-effector position, holding initial orientation3; 9 with variable stiffness; 15 with variable impedance
IK_POSEEnd-effector position and orientation (inverse kinematics)7
JOINT_POSITIONJoint positionsn; 2n with variable stiffness; 3n with variable impedance
JOINT_VELOCITYJoint velocitiesn
JOINT_TORQUEJoint torquesn

Expanded article table

Source: robosuite v1.5 white paper; n is the number of controlled joints [1].

For OSC_POSE, OSC_POSITION and JOINT_POSITION, the policy can use fixed impedance gains, choose the stiffness itself (with damping set automatically for a critically damped response), or choose both stiffness and damping, which lets a learned policy vary how compliant the robot is during contact [1]. The paper argues that torque-based controllers aid sim-to-real transfer, because torque control is common on real research arms such as the Sawyer, Panda, IIWA and Jaco [1].

Sensors and observations

Sensor readings include camera images, force-torque measurements from a sensor that every gripper model carries at the wrist, and pressure signals [1]. Joint positions and velocities are read as properties of the robot rather than through sensors, because MuJoCo computes them directly [1]. Camera images are produced by the renderer and added to the observation dictionary automatically for the cameras named when the environment is created [1]. The v1.2 release added observable sensor models and dynamics randomization [2].

Rendering

As of November 2026 the documentation describes two rendering paths. The default MuJoCo renderer, estimated at about 3,500 frames per second, uses MuJoCo's OpenGL context and is the renderer the environments and assets are tuned for [17]. The Isaac renderers, added in v1.5, require NVIDIA Isaac Sim and Isaac Lab and run only on Linux or Windows; they offer real-time ray tracing (estimated at about 58 frames per second) and slower path tracing (about 2.8 frames per second at 64 samples per pixel), which the documentation suggests for generating data offline [17].

Teleoperation and demonstrations

robosuite includes device drivers for human teleoperation. The v1.5 paper lists a keyboard, the 3Dconnexion SpaceMouse and MJGUI, which uses MuJoCo's own viewer and the mouse cursor to drag controller targets; the authors write that they used the SpaceMouse extensively for collecting demonstrations [1]. The documentation also lists a DualSense controller, and any device can be added by subclassing the Device class [1][4].

A script records human demonstrations with a chosen device and saves them as a demo.hdf5 file [18]. Each demonstration stores the task's MJCF model, the flattened MuJoCo simulator states over time and the actions taken. The documentation explains that storing states rather than observations saves disk space and lets users regenerate any observation type later, such as images from a different camera [18]. It also warns that replaying the recorded actions is deterministic only on the machine that recorded them, and recommends resetting the simulator to the recorded states to reproduce trajectories [18]. For learning from these datasets, the documentation points to robomimic, its sister project in the ARISE Initiative [2][18].

Benchmark

The benchmark tasks are the ones introduced with v1.0, each with randomized initial conditions [1].

TaskRobotsGoal
Block LiftingOne armLift a cube above a set height
Block StackingOne armPlace one cube on top of another
Pick-and-PlaceOne armMove four objects from a bin into matching containers (single-object variants exist)
Nut AssemblyOne armFit a square nut and a round nut onto matching pegs (single-nut variants exist)
Door OpeningOne armTurn a handle and open a door
Table WipingOne armErase markings from a whiteboard surface with an eraser mounted on the hand
Two Arm LiftingTwo armsLift a two-handled pot together while keeping it level
Two Arm Peg-In-HoleTwo armsInsert a peg held by one arm into a hole in a board held by the other
Two Arm HandoverTwo armsPick up a hammer with one arm and hand it to the other

Expanded article table

Source: robosuite v1.5 white paper [1].

The authors ran Soft Actor-Critic (SAC) on all nine tasks using an extended version of the rlkit library, low-dimensional state observations, Panda and Sawyer robots, and either OSC_POSE or JOINT_VELOCITY control [1]. Each agent was trained for 500 epochs with 500-step episodes over five random seeds; each run took about two days on 2 CPUs without a GPU [1]. SAC solved three tasks (Block Lifting, Door Opening and Two Arm Peg-In-Hole) and made slow progress on the rest, which the authors attribute to the need for better exploration over longer horizons [1]. In an ablation on Block Lifting and Door Opening, both robots learned faster with operational space control than with joint velocity control, which the authors hypothesize is due to faster exploration in task space [1]. The results were released in a separate benchmark repository [1].

Use in research

The white paper cites earlier uses of robosuite in reinforcement learning (SURREAL and work on variable impedance control in end-effector space), imitation learning (the RoboTurk crowdsourcing platform) and sim-to-real transfer [1]. Several later systems from the same group of researchers build on or use it:

ProjectVenueRelationship to robosuite
robomimicCoRL 2021Study of learning from offline human demonstrations; all of its simulation tasks were designed with MuJoCo and robosuite [14]. Its v0.4.0 release (March 2025) added support for robosuite v1.5 [20]
LIBERONeurIPS 2023 (Datasets and Benchmarks)Lifelong robot learning benchmark whose task-generation pipeline is built on robosuite [15]
MimicGenCoRL 2023Demonstration data-generation system; some of its evaluation tasks are implemented in robosuite and others in Factory, which runs on Isaac Gym [16]
RoboCasaRSS 2024Household simulation framework that adopts robosuite as its core platform and extends it to mobile manipulators, humanoids and quadrupeds with arms [12]. RoboCasa v0.2 moved to robosuite v1.5 as its backend [21]
DexMimicGenICRA 2025Data generation for bimanual dexterous manipulation; its code requires robosuite to be installed [22][23]

Expanded article table

The RoboCasa authors write that they chose robosuite "because of its focus on physical realism, high speed, and modular design" and inherited its environment model formats and robot controllers [12].

Availability

robosuite is distributed on PyPI as the robosuite package and on GitHub under the ARISE-Initiative organization [2][6]. The license file is the MIT License, with copyright held by the Stanford Vision and Learning Lab and the UT Robot Perception and Learning Lab [19]. As of November 2026 the GitHub repository had about 2,700 stars and 800 forks [2]. Documentation, including archived versions for v1.3 and v1.4, is hosted at robosuite.ai [2][4].

Continue reading

More to read

Topics:AI BenchmarksEmbodied AIOpen Source AIRobotics

References

  1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34 ^35 ^36Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany, Yifeng Zhu. "robosuite: A Modular Simulation Framework and Benchmark for Robot Learning" (v3). arXiv:2009.12293, 2025-01-18. arxiv.org/...2009.12293v3
  2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16ARISE Initiative. "robosuite" (README). GitHub. github.com/...robosuite
  3. ^1 ^2 ^3ARISE Initiative. "v1.0 major release". GitHub, 2020-09-28. github.com/...v1.0
  4. ^1 ^2robosuite. "Overview". robosuite documentation. robosuite.ai/...overview
  5. ^1 ^2Linxi Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei. "SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark". Proceedings of The 2nd Conference on Robot Learning, PMLR 87:767-782, 2018. proceedings.mlr.press/...fan18a
  6. ^1 ^2 ^3 ^4 ^5Python Package Index. "robosuite" (release history). PyPI. pypi.org/...robosuite
  7. ^1 ^2Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín. "robosuite: A Modular Simulation Framework and Benchmark for Robot Learning" (v1). arXiv:2009.12293, 2020-09-25. arxiv.org/...2009.12293v1
  8. ^1 ^2arXiv. "robosuite: A Modular Simulation Framework and Benchmark for Robot Learning" (abstract page and submission history). arXiv:2009.12293. arxiv.org/...2009.12293
  9. ^1 ^2 ^3ARISE Initiative. "robosuite 1.3.0 Release Notes". GitHub, 2021-10-19. github.com/...v1.3
  10. ^1 ^2ARISE Initiative. "robosuite 1.4.0 Release Notes". GitHub, 2022-11-30. github.com/...v1.4.0
  11. ^1 ^2 ^3ARISE Initiative. "Robosuite v1.5 Release Notes". GitHub, 2024-10-29. github.com/...v1.5.0
  12. ^1 ^2 ^3Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, Yuke Zhu. "RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots". RSS 2024; arXiv:2406.02523, 2024. arxiv.org/...2406.02523
  13. ^ARISE Initiative. "robosuite_models" (README). GitHub. github.com/...robosuite_models
  14. ^1 ^2Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, Roberto Martín-Martín. "What Matters in Learning from Offline Human Demonstrations for Robot Manipulation". CoRL 2021; arXiv:2108.03298, 2021. arxiv.org/...2108.03298
  15. ^1 ^2Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, Peter Stone. "LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning". NeurIPS 2023 Datasets and Benchmarks Track; arXiv:2306.03310, 2023. arxiv.org/...2306.03310
  16. ^1 ^2Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, Dieter Fox. "MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations". CoRL 2023; arXiv:2310.17596, 2023. arxiv.org/...2310.17596
  17. ^1 ^2robosuite. "Renderers". robosuite documentation. robosuite.ai/...renderers
  18. ^1 ^2 ^3 ^4robosuite. "Human Demonstrations". robosuite documentation. robosuite.ai/...demonstrations
  19. ^1 ^2ARISE Initiative. "LICENSE" (robosuite repository). GitHub. github.com/...LICENSE
  20. ^ARISE Initiative. "robomimic" (README). GitHub. github.com/...robomimic
  21. ^RoboCasa. "robocasa" (README). GitHub. github.com/...robocasa
  22. ^Zhenyu Jiang, Yuqi Xie, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlekar, Linxi Fan, Yuke Zhu. "DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning". ICRA 2025; arXiv:2410.24185, 2024. arxiv.org/...2410.24185
  23. ^NVlabs. "dexmimicgen" (README). GitHub. github.com/...dexmimicgen
  24. ^ARISE Initiative. "v1.5.2" (release notes). GitHub, 2025-12-24. github.com/...v1.5.2

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

2 revisions · v2 · 3,170 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent fact-check before publication on November 18, 2026: 112 claims checked against 24 cited and independent sources; 1 material and 6 minor corrections applied before publication.

Cite this page: AI Wiki. "robosuite." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/robosuite

Edit article