CtrlK
NVIDIA Isaac Sim is an open-source robotics simulation platform built on NVIDIA Omniverse for developing, testing, training, and deploying AI-powered robots in physically based virtual environments. It provides high-fidelity, GPU-accelerated physics simulation, photorealistic rendering, synthetic data generation, and integration with popular robotics frameworks such as ROS 2. Isaac Sim serves as a core component of NVIDIA's broader Isaac robotics platform, which encompasses tools for robot learning, perception, manipulation, and sim-to-real transfer.
Licensed under Apache 2.0 and freely available on GitHub, Isaac Sim is used by leading robotics companies, research institutions, and manufacturers worldwide to train robot policies, generate labeled datasets for computer vision models, build digital twins of factories and warehouses, and validate robot behavior before physical deployment.
The Isaac robotics platform traces its roots to NVIDIA's early investments in autonomous machine development. The first version of Isaac Sim, released in 2020 as a preview, was built on NVIDIA's Omniverse Kit and featured RTX graphics, PhysX 5.0 physics, and Python scripting for robot simulation. Earlier prototypes had experimented with Unreal Engine 4 and Unity as rendering backends, but NVIDIA ultimately standardized on Omniverse as the foundation for all future development.
Isaac Sim 2020.1, announced alongside the Isaac SDK 2020.1, focused on LIDAR-based navigation of the "Carter" reference robot and demonstrated the viability of GPU-accelerated robotics simulation with ray-traced sensor models.
Throughout 2021 and 2022, NVIDIA expanded Isaac Sim's capabilities significantly. Isaac Sim 2022.1, released in June 2022, introduced improvements aimed at bridging the simulation-to-reality gap for robotics developers. Isaac Sim 2022.2, released in December 2022, added a people simulator and conveyor belt utility to support warehouse logistics and manufacturing robotics use cases.
Isaac Sim 2023.1.0, released in October 2023, brought further refinements to the platform and represented the last version under the year-based naming convention. This release corresponded internally to version 4.0.0, marking the transition to a numeric versioning scheme.
Isaac Sim 4.0 launched on May 30, 2024, alongside Isaac Lab, a unified reinforcement learning platform that replaced several earlier frameworks. This release introduced support for humanoid robot models from companies including 1X Technologies, Unitree Robotics, and Boston Dynamics.
Isaac Sim 4.5, released in early 2025 following its announcement at CES 2025, added integration with the NVIDIA Cosmos world foundation model, improved URDF import workflows, enhanced physics simulation and modeling, and a new joint visualization tool.
Isaac Sim 5.0 achieved general availability at SIGGRAPH 2025 in August 2025, marking a milestone as the first fully open-source release on GitHub. Key additions included neural reconstruction and rendering via NVIDIA Omniverse NuRec, advanced synthetic data generation pipelines, new robot models, improved sensor simulation with a new OmniSensor USD schema, full ROS 2 Jazzy Jalisco support, and a new ZeroMQ bridge for simulator control.
Isaac Sim 5.1.0, released on October 21, 2025, updated the platform to Kit 107.3.3, introduced DGX Spark support, added new robot assets from Schunk, enhanced semantic segmentation for RTX sensors, and improved Docker multi-architecture support.
As of early 2026, Isaac Sim 6.0 is under active development as a pre-release developer build, featuring an architectural overhaul with multi-backend physics, a pluggable renderer system, and a kit-less installation mode.
Isaac Sim is built on NVIDIA Omniverse Kit, a modular toolkit for constructing 3D simulation applications. Omniverse Kit provides functionality through lightweight C-based plugins for performance and API stability, alongside a Python interpreter for scripting and customization. The architecture is designed to be fully extensible, allowing developers to build custom OpenUSD-based simulators or integrate Isaac Sim capabilities into existing testing and validation pipelines.
At the heart of Isaac Sim is OpenUSD (Universal Scene Description), an open-source 3D scene description format originally developed by Pixar. USD serves as the unifying data interchange layer, enabling interoperability between different tools, teams, and workflows. Isaac Sim can ingest data from multiple source formats and convert them into USD for simulation.
| Import Format | Description |
|---|---|
| URDF | Unified Robot Description Format, commonly used in ROS |
| MJCF | MuJoCo XML format for physics-based robot models |
| CAD | Computer-aided design files from engineering tools |
| OnShape | Cloud-based CAD models |
| NuRec | Neural reconstructions from real-world sensor captures |
USD provides both a human-readable text format (.usda) for direct editing and a binary-encoded format (.usd) for performance and storage optimization. SimReady 3D assets built in USD can be reused across tools and loaded directly into Isaac Sim, where USDPhysics colliders, rigid body dynamics, and composition-arc-based variants enable testing of robots in virtual facilities that closely mirror real-world operations.
Isaac Sim supports multiple physics backends for simulating robot dynamics and environmental interactions.
NVIDIA PhysX is the primary physics engine, providing GPU-accelerated simulation of rigid bodies, articulated multi-joint systems, vehicle dynamics, and SDF (signed distance field) colliders. For each discrete time step, PhysX advances simulation objects given their current state and additional inputs such as control-policy torques. The updated state is written back to USD, where it can be processed by the user, a reinforcement learning policy, or other extensions such as the Omniverse RTX Renderer.
NVIDIA Newton, announced in 2025, is a next-generation open-source, GPU-accelerated physics engine co-developed by Google DeepMind, Disney Research, and NVIDIA. Managed by the Linux Foundation and built on NVIDIA Warp, Newton features differentiable physics for gradient propagation through simulation, making it particularly suitable for optimization-based robot learning. Newton is licensed under Apache 2.0 and is integrated into Isaac Lab 3.0 Beta as an experimental feature.
| Physics Engine | License | Key Features |
|---|---|---|
| NVIDIA PhysX | Proprietary (bundled) | GPU-accelerated rigid body, articulation, vehicle dynamics, SDF colliders |
| NVIDIA Newton | Apache 2.0 | Differentiable physics, GPU-accelerated via Warp, multi-physics, OpenUSD support |
Isaac Sim uses NVIDIA's RTX rendering technology for photorealistic visualization. The rendering pipeline leverages OptiX and RTX hardware for path tracing and ray tracing, with MDL-based (Material Definition Language) physically based rendering materials that enable spectral, BRDF-accurate visualization. This level of rendering fidelity is critical for generating realistic synthetic data for training computer vision and perception models.
In Isaac Sim 5.0, NVIDIA introduced neural reconstruction and rendering through the Omniverse NuRec libraries, which convert real-world images into interactive 3D Gaussian Splatting-based simulations. This allows developers to create photorealistic digital environments directly from sensor captures of real-world locations.
Isaac Sim provides a comprehensive suite of simulated sensors that generate physically accurate data streams matching those of real-world hardware. These sensor models are essential for developing and testing robot perception pipelines in simulation before deploying to physical robots.
| Sensor Type | Simulation Method | Key Capabilities |
|---|---|---|
| RGB Camera | RTX ray-traced rendering | Photorealistic images, configurable resolution and field of view |
| Depth Camera | RTX-based depth computation | Stereo depth modeling, realistic noise profiles |
| RTX Lidar | GPU ray casting at render time | Rotary and solid-state configurations, pre-made configs for commercial sensors |
| IMU | Physics-based integration | 6-axis inertial measurement, device-side processing |
| Stereo Camera | Dual-sensor RTX rendering | ZED X and Hawk camera models with paired RGB and IMU |
| Contact Sensor | Physics collision detection | Force and torque measurements at contact points |
RTX Lidar sensors are simulated on the GPU using RTX hardware at render time. Isaac Sim ships with a library of pre-configured Lidar profiles approximating commercial sensors from publicly available specifications. The platform also supports the OmniSensor USD schema introduced in version 5.0 for standardized sensor definitions.
Isaac Sim 5.1.0 added semantic segmentation support for RTX sensors and performance improvements for RTX Sensor Annotators, further enhancing the fidelity of sensor simulation for perception development.
One of Isaac Sim's most powerful capabilities is its synthetic data generation (SDG) pipeline, powered by the Omniverse Replicator extension. Replicator allows developers to create large, diverse, and fully annotated datasets for training deep learning models without the cost and time of manual data collection and labeling.
Domain randomization is a core technique in Isaac Sim's SDG pipeline. By introducing variability into simulated environments, domain randomization helps bridge the gap between simulation and reality, producing models that generalize better to real-world conditions. Replicator supports randomization of:
A key advantage of Replicator is its ability to perform "on-the-fly" randomization without reloading or re-parsing 3D assets. This significantly reduces computational overhead and speeds up data generation compared to approaches that require full scene reconstruction for each variation.
Replicator generates ground-truth annotations alongside rendered images, supporting multiple annotation formats commonly used in computer vision research and production.
| Annotation Type | Description |
|---|---|
| RGB | Standard color image data |
| 2D Bounding Box | Rectangular bounds around objects in image space |
| 3D Bounding Box | Oriented bounding volumes in world coordinates |
| Semantic Segmentation | Per-pixel class labels |
| Instance Segmentation | Per-pixel object instance labels |
| Depth | Per-pixel distance from camera |
| Surface Normals | Per-pixel surface orientation |
Annotated datasets can be exported in COCO and KITTI formats for direct use with popular training frameworks. The pipeline supports both object detection and pose estimation workflows.
Starting with Isaac Sim 4.5, the platform integrates with NVIDIA Cosmos, a world foundation model that can augment synthetic data with enhanced photorealism. Developers compose SimReady 3D scenes in Isaac Sim, render images or videos of specific robot tasks, and then use Cosmos Transfer to generate photoreal video clips for retraining perception and policy models. Omniverse Replicator includes a dedicated writer optimized for Cosmos Transfer input, streamlining this multi-stage data pipeline.
Isaac Lab is a GPU-accelerated, open-source framework for robot learning built on top of Isaac Sim. It provides a unified platform for training robot policies using reinforcement learning, imitation learning, and other methods at scale.
Isaac Lab's development originated from the Orbit framework, which was jointly developed by NVIDIA, the AI Institute, ETH Zurich, and the University of Toronto. Isaac Lab replaces several earlier frameworks:
Isaac Lab was announced alongside Project GR00T in March 2024 and serves as the successor to NVIDIA Isaac Gym.
Isaac Lab's modular architecture and NVIDIA GPU-based parallelization enable running thousands of parallel simulation environments simultaneously, dramatically accelerating the training process. Key features include:
Isaac Lab 2.0 shipped alongside Isaac Sim 4.5 in early 2025. Isaac Lab 2.2, released at SIGGRAPH 2025 with Isaac Sim 5.0, added enhancements including cloud deployment on NVIDIA Brev and an open-source policy evaluation framework developed in collaboration with Lightwheel.
Isaac Lab 3.0 Beta, built on Isaac Sim 6.0, introduces a ground-up architectural overhaul with multi-backend physics (including Newton), a pluggable renderer system, Warp-native data pipelines, and a kit-less installation mode.
Humanoid robot developers using Isaac Lab include 1X Technologies, Agility Robotics, Berkeley Humanoid, Boston Dynamics, Fourier Intelligence, Galbot, Unitree Robotics, and XPENG Robotics.
Isaac GR00T (Generalist Robot 00 Technology) is NVIDIA's research initiative and open development platform for general-purpose robot foundation models, with a primary focus on humanoid robotics. Isaac Sim serves as the simulation backbone for GR00T model training and evaluation.
The latest release, Isaac GR00T N1.6, is a reasoning vision-language-action model for robot skills that integrates NVIDIA Cosmos Reason, an open reasoning vision language model built for physical AI. Cosmos Reason acts as the robot's planning layer, converting vague natural language instructions into step-by-step action plans.
GR00T models are trained using a combination of imitation learning, reinforcement learning, and video data within GPU-accelerated Isaac Sim environments.
GR00T-Dreams is a blueprint for generating vast amounts of synthetic motion data to teach robots new behaviors. NVIDIA Research used GR00T-Dreams to generate synthetic training data for GR00T N1.5 in approximately 36 hours, compared to what would have required nearly three months of manual human data collection.
GR00T-Mimic provides data pipelines for collecting high-quality robot demonstration data from both real-world and simulated environments via Isaac TeleOp.
Companies evaluating Isaac GR00T N models include AeiROBOT, Franka Robotics, LG Electronics, Lightwheel, Mentee Robotics, Neura Robotics, Solomon, Techman Robot, and UCR.
Isaac Sim provides deep integration with the Robot Operating System (ROS), the most widely used middleware framework in robotics. The ROS 2 Bridge extension connects Isaac Sim to ROS 2, enabling bidirectional communication of sensor data, control commands, and transforms.
Developers can use Isaac Sim as a drop-in replacement for physical hardware during ROS 2 development, running full navigation and manipulation stacks against simulated robots and environments. This enables rapid iteration on perception, planning, and control algorithms without the constraints of physical testing.
Isaac ROS is a complementary collection of CUDA-accelerated ROS 2 packages that run on NVIDIA Jetson and discrete GPUs. It includes packages for visual SLAM (cuVSLAM), depth perception, 3D reconstruction (nvblox), and motion planning (cuMotion). While Isaac ROS is designed for deployment on physical robots, Isaac Sim provides the simulation environment for developing and testing Isaac ROS pipelines before real-world deployment.
Isaac Sim supports advanced robot manipulation workflows through integration with NVIDIA's CUDA-accelerated motion planning libraries.
NVIDIA cuMotion is a CUDA-accelerated library for computing optimal-time, minimal-jerk trajectories for serial robot arms. It solves motion planning problems at scale by running multiple trajectory optimizations simultaneously and returning the best solution. cuMotion can avoid collisions with obstacles represented as cuboids, meshes, signed distance fields (computed from depth image streams using nvblox), or any combination of the three.
On discrete GPUs such as the RTX 6000 (Ada Generation), cuMotion computes collision-free trajectories in tens of milliseconds. Trajectories can be planned and executed on simulated robots in Isaac Sim, allowing rapid development iteration without physical hardware.
NVIDIA Isaac Manipulator is a comprehensive solution that combines several GPU-accelerated tools for robotic arm development:
| Component | Function |
|---|---|
| cuMotion | Collision-free motion planning and trajectory optimization |
| nvblox | Real-time 3D scene reconstruction from depth sensors |
| FoundationPose | 6D object pose estimation |
| FoundationStereo | Stereo depth estimation |
| SyntheticaDETR | Object detection |
Isaac Manipulator accelerates the development of pick-and-place, assembly, and inspection workflows by providing AI-powered perception and planning capabilities that work seamlessly within Isaac Sim for simulation and on physical robots via Isaac ROS.
NVIDIA Isaac Perceptor is a collection of CUDA-accelerated libraries, AI models, and reference workflows for developing autonomous mobile robots (AMRs). Built on ROS 2, it enables robots to perceive, localize, and navigate in unstructured environments such as warehouses and factories.
Key capabilities include:
Isaac Perceptor pipelines can be developed and tested entirely within Isaac Sim using simulated cameras, lidar, and IMU sensors before deployment on physical AMRs running on NVIDIA Jetson hardware.
Isaac Sim includes prebuilt 3D warehouse environments and practical scenarios for autonomous navigation. The platform supports simulation of conveyor belt systems, automated guided vehicles (AGVs), and AMRs operating in logistics facilities. Companies use Isaac Sim to optimize fleet management, path planning, and goods-to-person workflows before deploying systems in physical warehouses.
Isaac Sim enables building intelligent factory solutions that support design, simulation, and optimization of industrial assets and processes. Use cases include assembly line simulation, quality inspection with machine vision, robot work cell design, and multi-robot coordination.
Isaac Sim serves as a key tool for creating operational digital twins of physical facilities. By combining accurate physics simulation, photorealistic rendering, and real-time data integration, manufacturers can mirror their production environments in simulation. Digital twins built with Isaac Sim enable predictive maintenance, process optimization, and "what-if" scenario testing without disrupting physical operations.
Numerous organizations across industries have adopted NVIDIA Isaac Sim for robotics development and deployment.
| Company | Application |
|---|---|
| Amazon Robotics | Digital twins of warehouses; optimizing and testing the Proteus AMR |
| BMW Group | Training Smart Transport Robot (STR) and SortBot for factory logistics; multi-modal AI including motion planning, object detection, and pose estimation |
| Mercedes-Benz | Virtual factory digital twins for testing humanoid robot deployment; reducing factory planning cycles by approximately two years |
| Foxconn | Simulating screw tightening, cable insertion, and other assembly tasks; digital twin factory layouts with Fii Digital Twin platform |
| KION Group | Advancing autonomous warehouse systems using Omniverse-based digital twins with Accenture |
| PepsiCo | Evaluating physical AI capabilities for logistics optimization |
| Siemens | Partnership with NVIDIA to build an industrial AI operating system integrating Omniverse and Isaac Sim |
| Universal Robots | AI Accelerator developed on the NVIDIA Isaac platform for handling complex tasks in dynamic environments |
| Amazon Lab126 | Implementing Isaac Sim and Isaac Lab for robotics development |
| Figure AI | Using Isaac frameworks for humanoid robot development |
Isaac Sim requires specific hardware and software configurations to run effectively.
| Component | Minimum | Recommended |
|---|---|---|
| GPU | NVIDIA RTX 4080 or A40 | NVIDIA RTX 5080 or L40S |
| VRAM | 16 GB | 16-48 GB |
| CPU | Multi-core x86_64 or ARM (aarch64) | High-performance multi-core |
| RAM | 32 GB | 64 GB or more |
GPUs without RT Cores (such as the A100 and H100, which are designed for compute workloads) are not supported because Isaac Sim relies on RTX ray tracing for rendering and sensor simulation.
| Platform | Supported Versions |
|---|---|
| Linux | Ubuntu 22.04 (x86_64 and aarch64); Ubuntu 24.04 requires GCC/G++ 11 |
| Windows | Windows 10/11 (64-bit) |
| Python | 3.11 |
| NVIDIA Driver (Linux) | 580.65.06 or later |
| NVIDIA Driver (Windows) | 580.88 or later |
Isaac Sim supports multiple deployment configurations:
The Isaac Sim Compatibility Checker, a lightweight utility integrated into Isaac Sim 5.1.0, programmatically validates system requirements before installation.
| Version | Release Period | Notable Features |
|---|---|---|
| 2020.1 | 2020 | First preview; Omniverse Kit foundation, RTX graphics, PhysX 5.0 |
| 2022.1 | June 2022 | Improved sim-to-real capabilities |
| 2022.2 | December 2022 | People simulator, conveyor belt utility, warehouse logistics support |
| 2023.1.0 (4.0.0) | October 2023 | Transition to numeric versioning; Isaac Lab launch |
| 4.0 | May 2024 | Isaac Lab unified RL platform; humanoid robot support; new robot assets |
| 4.1 | 2024 | Ackermann controller; improved ROS 2 support |
| 4.2 | 2024 | RTX-based tiled rendering; deformable environments |
| 4.5 | February 2025 | Cosmos integration; improved URDF import; joint visualization |
| 5.0 | August 2025 | Open source on GitHub (Apache 2.0); neural reconstruction; NuRec; OmniSensor schema; ROS 2 Jazzy |
| 5.1 | October 2025 | Kit 107.3.3; DGX Spark support; Schunk robot assets; semantic segmentation for RTX sensors |
| 6.0 (dev) | In development | Multi-backend physics; pluggable renderer; kit-less mode; Newton physics integration |
Isaac Sim source code is released under the Apache 2.0 open-source license, making it free to use for development, research, and commercial applications. The source code is available on GitHub at the isaac-sim/IsaacSim repository.
It is important to note that while the Isaac Sim extensions and application code are open source, building or running the platform requires additional components (such as the Omniverse Kit SDK, 3D model assets, and textures) that are covered under separate NVIDIA license terms.