NVIDIA Isaac for Healthcare
NVIDIA Isaac for Healthcare is NVIDIA's open-source developer framework for building, simulating, training, and deploying medical robots. It was announced on 18 March 2025 at NVIDIA's GTC conference and is distributed on GitHub under the Apache 2.0 license as a domain-specific layer on top of NVIDIA Isaac Sim, Isaac Lab, NVIDIA Omniverse, and NVIDIA Holoscan.[1][3][4] On 22 July 2026 NVIDIA added the Medical Physics Simulation framework, a GPU-native package for modelling how instruments interact with anatomy, and published its technical design on 28 July 2026.[2][5]
| Field | Value |
|---|---|
| Developer | NVIDIA |
| Announced | 18 March 2025 (GTC 2025)[3] |
| Type | Healthcare robotics development framework |
| Code license | Apache 2.0 (i4h-workflows, i4h-sensor-simulation, i4h-asset-catalog)[5] |
| Model weights license | NVIDIA Open Model License (Cosmos-H family)[9] |
| Latest release surveyed | i4h-workflows v0.7.1, 23 July 2026[6] |
| Built on | Isaac Sim, Isaac Lab, Omniverse, Holoscan, MONAI, NVIDIA Warp, Newton, NVIDIA Cosmos[2][4] |
| Repository | github.com/isaac-for-healthcare[4] |
Why healthcare robotics needs its own simulator
Autonomous driving and warehouse robotics can be improved by collecting more data. Surgical and interventional robotics cannot. Every demonstration needs specialised hardware, clinical expertise, and access to a patient or a laboratory. NVIDIA's engineering team frames the resulting bottleneck as three problems: a data gap (most teams hold hundreds of demonstrations rather than the tens of thousands needed), a generalisation problem (imitation learning plateaus at the edge of its training distribution, and the clinically important cases are precisely the rare ones), and slow development velocity, since benchtop phantoms, cadaver studies, and animal models are sequential and expensive. NVIDIA puts total medical robotics development cycles at four to seven years.[1]
That is the argument for simulation: a policy can attempt a manoeuvre a million times in silico at no clinical risk, and rare anatomies can be synthesised rather than waited for. The counter-argument, which the same engineers state plainly, is that a simulator is only useful to the extent that its physics are trustworthy.
Platform architecture
Isaac for Healthcare is organised around what NVIDIA calls its three-computer framework for physical AI: one computer trains models, one runs the simulation and digital twin, and one executes the trained policy on the robot in real time.[3]
| Layer | Component | Function |
|---|---|---|
| Training | MONAI, including the MAISI and VISTA-3D models | Pretrained medical models; AI-assisted CT synthesis and segmentation for building anatomy[3] |
| Simulation | NVIDIA Omniverse, Isaac Sim, Isaac Lab | Physically accurate virtual environments, synthetic data generation, GPU-parallel policy training[3][4] |
| Runtime | NVIDIA Holoscan | On-robot deployment, real-time multi-modal sensor processing, hardware-in-the-loop testing[3][4] |
The framework's stated scope is broader than surgery: therapeutics (surgical robotics, interventional radiology, radiotherapy), diagnostics (ultrasound, CT, cone-beam CT), hospital and patient care robots, and rehabilitation and assistive systems.[4] Anatomy is assembled into a single OpenUSD file so that organs, instruments, and robots share one scene description.[3] NVIDIA's product page groups the deliverables into five categories: medical sensor simulation, ready-to-use models and policies (including post-trained PI0 and GR00T N1 checkpoints), reference workflows, synthetic data generation, and a catalogue of simulation-ready medical and anatomical assets.[13]
Reference workflows
The public i4h-workflows repository ships seven end-to-end reference implementations as of v0.7.1.[5]
| Workflow | Description |
|---|---|
| Robotic Surgery | Physics-based surgical robot simulation supporting the da Vinci Research Kit (dVRK), dual-arm configurations, and STAR arms, with GPU-parallelised reinforcement learning |
| Robotic Ultrasound | Autonomous ultrasound imaging with GPU raytraced acoustic sensor simulation generating B-mode images; supports PI0 and GR00T N1 policies |
| Telesurgery | Remote operation with hardware-accelerated H.264/HEVC video, RTI DDS transport, and x86/AARCH64 deployment |
| SO-ARM Starter | Surgical assistant robotics on the SO-ARM101 manipulator using a GR00T N1.5 diffusion policy |
| Rheo | Smart hospital digital twin blueprint combining physical GR00T agents with vision-language monitoring agents |
| Agentic | Configurable Isaac Lab Arena environments for manipulation, loco-manipulation, trocar assembly, and ultrasound scanning |
| Catheter Navigation (Endoluminal) | Endovascular catheter and guidewire navigation with a differentiable fluoroscopy renderer and an XPBD rod solver |
The repository was created in February 2025 and its first tag, v0.1.0ea, was published on 18 March 2025, the same day as the announcement. Releases have arrived roughly every two to three months since: v0.2.0 (July 2025), v0.3.0 (September 2025), v0.4.0 (October 2025), v0.5.0 (March 2026), v0.6.0 (June 2026), and v0.7.0 on 21 July 2026, which introduced the Endoluminal Workflow one day before NVIDIA announced Medical Physics Simulation publicly.[6]
Medical Physics Simulation
NVIDIA announced the Medical Physics Simulation framework on 22 July 2026 in a post by David Niewolny, describing it as the first open-source GPU-accelerated framework of its kind. MassDevice reported that it was unveiled at the SRS 2026 meeting.[2][19] The framework contains two classical solvers and a generative branch, and NVIDIA argues that open weights matter specifically for medicine: developers who can inspect the data, models, and weights can "reproduce results, evaluate performance across different anatomies and scenarios, identify limitations and build evidence for regulatory review."[2]
| Module | Status | What it simulates | Method |
|---|---|---|---|
| Endoluminal Simulation | General availability | Long flexible instruments (catheters, guidewires) in endoluminal cavities under fluoroscopic guidance | Cosserat rods solved with extended position-based dynamics (XPBD)[1] |
| Surgical Simulation | Early access | Soft-tissue deformation, grasping, cutting, clipping, diathermy, haptics | Tetrahedral meshes with position-based dynamics under distance and volume constraints[1] |
| Cosmos-H generative simulation | Released July 2026 | Surgical video conditioned on robot kinematics | Distilled few-step autoregressive video diffusion[1][8] |
The endoluminal solver
Both classical modules are written in Python using NVIDIA Warp and Newton Physics.[1] Flexible instruments are modelled as Cosserat rods, a formulation from continuum mechanics that represents a one-dimensional elastic body capable of bending, twisting, and stretching. Standard XPBD handles constraints through local iterative projection, which is a poor fit for a metre-long catheter: it can take many iterations for a nudge at the surgeon's end of the device to propagate to the tip inside a vessel.
NVIDIA's solver instead assembles the coupled rod constraints into a matrix. Each rod segment contributes six constraint equations, three for stretch and shear and three for bending and twisting, which produces a block-tridiagonal system of 6 by 6 blocks. The Thomas algorithm, a specialised Gaussian elimination for tridiagonal matrices, solves that system in time linear in the instrument's length, and independent instruments run concurrently across vectorised environments on the GPU. The result is that insertion, hub rotation, and distal steering propagate along the whole instrument within a single simulation step.[1]
Contact with the vessel is handled by signed closest-point queries evaluated directly against the patient-specific triangle mesh, avoiding a precomputed distance field. Samples that penetrate the wall are projected back into the lumen while keeping tangential sliding. NVIDIA states that the current release supports rigid one-way containment only, and that two-way coupling to deformable vessel walls is planned for a future release.[1]
The module keeps a common API with other Newton solvers, and Torch-to-Warp interoperability through wp.from_torch and wp.to_torch keeps the control and reinforcement-learning interface zero-copy and GPU-resident. Isaac Lab handles environment orchestration and rollout control while the solver returns device-resident tensors for observations, rewards, and actions.[1] In the shipped code the solver appears as XPBDRodSolver, extended by XCathRodSolver for vessel-mesh containment via SDF and BVH broadphase, plus a hybrid NewtonXPBDRodSolver; a Slang-based differentiable digitally reconstructed radiograph renderer composites catheter attenuation into the fluoroscopic image using Beer-Lambert absorption.[7]
The surgical solver
The Surgical Simulation Module, in early access, represents soft tissue as tetrahedral meshes and solves deformation with position-based dynamics using distance-preserving and volume-preserving constraints. Corrections are accumulated and averaged on the GPU in Jacobi fashion, which removes constraint-ordering artefacts and parallelises cleanly. Deformation, collision queries, grasping, thermal diffusion, topology updates, and surface reconstruction all stay on the GPU; CUDA graph capture reduces kernel-launch overhead and CUDA interop passes data straight to the renderer. Rather than a monolithic engine, the module exposes an explicitly ordered sequence of simulation systems that each procedure configures.[1] Its initial release covers an end-to-end cholecystectomy: dissection of the hepatocystic triangle, clipping and cutting the cystic duct and artery, and separating the gallbladder from the liver.[1]
Reported performance
Every figure below is NVIDIA's own; none has been independently reproduced in a source examined here.
| Measurement | Reported result | Source |
|---|---|---|
| Endoluminal physics, single environment | About 1,300 Hz | Developer blog, 28 July 2026[1] |
| Endoluminal physics, 512 parallel environments | 60 Hz | Developer blog[1] |
| Full simulation plus rendering at 256 by 256 pixels | 63 frames per second | Developer blog[1] |
| Surgical module, consumer-grade GPU | Above 30 frames per second with eight physics substeps and eight deformation constraint iterations | Developer blog[1] |
| Cosmos-H-Dreams interactive generation | Above 30 frames per second | Developer blog[1] |
| Policy training across 8,192 parallel environments | From over five hours to under two minutes | NVIDIA corporate blog[2] |
The last row is the number NVIDIA led with publicly and the one most secondary coverage repeated. It is worth separating from the rest: the 1,300 Hz and 63 FPS figures come from a described experiment (512 environments, 1,500 training iterations) in the endoluminal module, whereas the 8,192-environment result is presented as a benchmark without the workload being specified. The same corporate post elsewhere says the framework "can run hundreds of parallel simulation environments," which sits awkwardly beside 8,192.[1][2]
Generative simulation and the Cosmos-H family
The second branch replaces hand-authored physics with learned dynamics. A generative world model is trained on video plus conditioning signals and predicts the next frames; because the output lives in the same visual space a camera or a policy sees, it can reproduce scene detail that would be impractical to author by hand. NVIDIA is explicit about the trade: such predictions "can be visually realistic without providing the explicit physical guarantees of a numerical solver."[1]
| Model | Conditioning | Purpose |
|---|---|---|
| Cosmos-H-Surgical-Predict | Initial image plus text | Forecasts plausible future surgical video[1] |
| Cosmos-H-Surgical-Transfer | Depth, edges, segmentation, or blur | Controlled domain transfer[1] |
| Cosmos-H-Surgical-Simulator | Surgical robot kinematics | Action-conditioned future prediction; released March 2026[1][10] |
| Cosmos-H-Dreams | Live action stream | Interactive real-time simulator distilled from the Simulator; released 23 July 2026[8][9] |
Cosmos-H-Surgical-Simulator is built on the public Cosmos-Predict2.5-2B world foundation model and driven by a 44-dimensional action vector encoding end-effector poses and gripper commands, unified across nine embodiments so that one set of weights serves multiple robots.[10] Cosmos-H-Dreams distils a bidirectional teacher warm-started from that checkpoint into a causal few-step student with a streaming key-value cache, generating in blocks of 12 frames at 288 by 512 resolution with two to four denoising steps per latent frame. The released checkpoint specialises in dVRK tabletop suturing, and NVIDIA validated real-time interactive serving on a single RTX PRO 6000, with A100 and H100 also supported. It is served over WebRTC and can be driven from a browser keyboard or a Meta Quest headset in virtual reality, or queried in closed loop by a learned policy.[8][9] Source code is Apache 2.0; the weights carry the NVIDIA Open Model License.[8][9]
Training data comes from Open-H, a community dataset NVIDIA announced at GTC 2026. NVIDIA's GTC post describes it as built with 35 collaborators and comprising 776 hours of surgical video, 11 robotic embodiments, and four surgical indications; the Hugging Face dataset card states 750 hours, 120,000 paired video-kinematics trajectories, and 4.5 TB; the Cosmos-H model cards describe the training mixture as roughly 26,500 demonstrations across 32 datasets, 9 embodiments, and more than 10 institutions, totalling about 4.9 million synchronised frames.[9][10][11][12] These are overlapping but not identical accountings, and NVIDIA has not reconciled them publicly. Notably, the dVRK post-training mixture deliberately retains failure and out-of-distribution episodes (dropped needles, missed throws, failed knots) so that the model renders the consequences of bad actions rather than only clean expert behaviour.[9]
The underlying stack
| Component | What it is | Role here |
|---|---|---|
| NVIDIA Warp | Python framework that JIT-compiles Python functions into differentiable CPU or GPU kernels, Apache 2.0[15] | Implementation language of both classical solvers[1] |
| Newton | Open GPU physics engine built on Warp and OpenUSD, co-developed by Disney Research, Google DeepMind, and NVIDIA, contributed to the Linux Foundation on 29 September 2025[14] | Physics backend; the endoluminal module keeps a common API with other Newton solvers[1] |
| Isaac Lab | GPU-parallel robot learning framework | Environment orchestration and rollout control for RL training[1] |
| Isaac Sim / Omniverse | Scene authoring and RTX rendering on OpenUSD | Digital twins of anatomy, instruments, and operating rooms[3][4] |
| NVIDIA Cosmos | World foundation models for physical AI | Base models post-trained into the Cosmos-H family[1] |
| Holoscan | Real-time sensor-processing SDK | On-robot runtime and hardware-in-the-loop testing[3][4] |
Simulation as regulatory evidence
The most consequential claim NVIDIA makes is not about frame rates. It is that open, inspectable simulation can help developers "build evidence for regulatory review."[2] That connects the framework to an existing regulatory concept rather than inventing one.
In the United States, the FDA finalised a guidance on 16 November 2023 titled "Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions" (Docket FDA-2021-D-0980). It sets out a risk-informed framework for judging how much confidence a computational model deserves, scaled to how heavily a submission relies on it and how bad it would be if the model were wrong. Crucially for this article, the guidance covers first-principles physics-based and mechanistic models and explicitly excludes statistical and data-driven models, including machine learning. It also aligns with the FDA-recognised standard ASME V&V 40, "Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices," published in 2018.[16][18] A separate voluntary FDA route, the Medical Device Development Tools (MDDT) programme, qualifies tools (including non-clinical computational models) for a specified context of use, so that a sponsor using a qualified tool within that context does not have to re-establish its suitability in each submission.[17]
Two cautions follow. First, none of this means Isaac for Healthcare or Medical Physics Simulation is FDA-cleared or FDA-qualified: no such clearance or MDDT qualification has been announced, and the frameworks are development tools, not devices. Second, the generative half of the framework falls outside the scope of the 2023 CM&S guidance by construction, since Cosmos-H is a learned model rather than a first-principles one. Of the named adopters, only Inner Logic is described by NVIDIA as producing "in silico evidence to support regulatory pathways."[2] Simulation of any kind supplements, and does not replace, bench, animal, and clinical evaluation.
Adoption
| Organisation | Reported use | Date |
|---|---|---|
| CMR Surgical (with Cambridge Consultants, part of Capgemini) | Cosmos-H-Dreams for implicitly learned interaction physics in soft-tissue procedures; contributed close to 500 hours of anonymised Versius clinical data to Open-H | July 2026[2] |
| Johnson & Johnson MedTech | Medical Physics Simulation plus a Cosmos-based foundation model to build digital twins of the endoluminal MONARCH urology platform, including kidney-stone scenarios | July 2026[2] |
| XCath | Endovascular autonomy policy training; earlier, digital twins of its endovascular robot and human vasculature | March 2025, July 2026[2][3] |
| Inner Logic | Synthetic data for validating device mechanics and producing in silico regulatory evidence | July 2026[2] |
| Medtronic Structural Heart | Exploring Medical Physics Simulation with simulated X-ray sensing for catheter navigation research | July 2026[2] |
| PeritasAI | Isaac for Healthcare and Rheo to train humanoid robots and VLA models for operating rooms, with Lightwheel and Advent Health Hospitals | March 2026[12] |
| Proximie | Cosmos-H to train multimodal vision-language models on operating room and intraoperative video | March 2026[12] |
| Moon Surgical | Prototyping autonomous robot setups adapting to surgeon technique | March 2025, March 2026[3][12] |
| Virtual Incision | Evaluating the platform for surgical synthetic data generation | March 2025[3] |
| Neptune Medical | Omniverse and Isaac Sim to design and simulate robotic endoscopy | March 2025[3] |
| ImFusion | Integrating Isaac for Healthcare into the ImFusion Suite | March 2025[3] |
| Kinova, Franka | Supplying simulation-ready robotic arms | March 2025[3] |
| Rob Surgical | Adopter of NVIDIA healthcare robotics tools | March 2026[12] |
MassDevice additionally reported that Karl Storz's Asensus and Stereotaxis have ongoing surgical robotics collaborations with NVIDIA, without specifying Isaac for Healthcare involvement.[19]
Limitations and open questions
The framework's own documentation names several gaps. Vessel walls are currently rigid, with one-way containment; two-way coupling to deformable vessels is future work.[1] The surgical module is early access and covers one procedure. Cosmos-H-Dreams ships a checkpoint specialised to dVRK tabletop suturing rather than in-vivo surgery, and the Cosmos-H-Dreams repository documents a known failure mode where the WebRTC stream flickers when the GPU is saturated and the hardware encoder cannot keep up.[8]
The deeper limit is the sim-to-real gap. Soft tissue is heterogeneous, anisotropic, perfused, and different in every patient, and a tetrahedral mesh with distance and volume constraints is a coarse model of it. The generative branch makes the tension explicit: Cosmos-H produces video that looks right without guaranteeing that the underlying mechanics are right, which is exactly the property that makes it useful for perception training and risky as a proxy for physical validation. NVIDIA's stated position is that the two approaches are complementary, with classical solvers supplying determinism and quantitative assessment and generative models supplying visual diversity at scale.[1] No independent validation of either branch against real tissue measurements appeared in the sources reviewed here.
Reception
Coverage in the week after 22 July 2026 was largely descriptive and drawn from NVIDIA's own announcement. HIT Consultant and TechTimes both led with the 8,192-environment training figure, the latter converting it into "a roughly 150-times speedup"; MassDevice, writing for a medical device rather than a computing audience, omitted that benchmark entirely and reported only that the framework "can run hundreds of parallel simulation environments."[19][21][22] The most sceptical framing came from TNW, which noted that "a simulation is also not a patient," that a policy that performs in a digital twin still has to prove itself in an operating theatre, and that "regulators have yet to say how much synthetic evidence they will accept." TNW also observed the commercial logic of the release: open-sourcing the tooling positions NVIDIA as the default infrastructure beneath the next generation of surgical robots, all of it running on NVIDIA GPUs.[20] No peer-reviewed independent evaluation of the framework had appeared as of 31 July 2026.
See also
- NVIDIA Isaac Sim
- Isaac Lab
- NVIDIA Newton
- NVIDIA Warp
- NVIDIA Cosmos
- NVIDIA Holoscan
- Surgical robot
- AI in healthcare
References
- ^Cristiana Dinea, Przemyslaw Korzeniowski, Javier Gamazo Tejero, Lukas Zbinden, Max Allan and Mostafa Toloui, "Developing Healthcare Robotics with GPU-Native Medical Physics Simulation", NVIDIA Technical Blog, 28 July 2026. developer.nvidia.com/...medical-physics-simulation
- ^David Niewolny, "NVIDIA Open Sources First GPU-Accelerated Medical Physics Simulation Framework", NVIDIA Blog, 22 July 2026. blogs.nvidia.com/...physics-simulation-open-source
- ^Mostafa Toloui, Andres Diaz-Pinto and Masoud Moghani, "Introducing NVIDIA Isaac for Healthcare, an AI-Powered Medical Robotics Development Platform", NVIDIA Technical Blog, 18 March 2025. developer.nvidia.com/...otics-development-platform
- ^"NVIDIA Isaac for Healthcare", GitHub organisation profile, accessed 31 July 2026. github.com/isaac-for-healthcare
- ^"Isaac for Healthcare - Workflows", `i4h-workflows` README, GitHub, accessed 31 July 2026. github.com/...i4h-workflows
- ^"Releases: isaac-for-healthcare/i4h-workflows", GitHub, accessed 31 July 2026. github.com/...releases
- ^"Endoluminal Workflow", `i4h-workflows/workflows/catheter_navigation` README, GitHub, accessed 31 July 2026. github.com/...catheter_navigation
- ^"Cosmos-H-Dreams", repository README, GitHub, accessed 31 July 2026. github.com/...Cosmos-H-Dreams
- ^"nvidia/Cosmos-H-Dreams", model card, Hugging Face, released 23 July 2026. huggingface.co/...Cosmos-H-Dreams
- ^"nvidia/Cosmos-H-Surgical-Simulator", model card, Hugging Face, released March 2026. huggingface.co/...Cosmos-H-Surgical-Simulator
- ^"nvidia/PhysicalAI-Robotics-Open-H-Embodiment", dataset card, Hugging Face, accessed 31 July 2026. huggingface.co/...calAI-Robotics-Open-H-Embodiment
- ^"NVIDIA GTC 2026: Live Updates on What's Next in AI", NVIDIA Blog, March 2026, section "NVIDIA Releases Open Physical AI Models for Healthcare Robotics". blogs.nvidia.com/...gtc-2026-news
- ^"NVIDIA Isaac for Healthcare", NVIDIA Developer product page, accessed 31 July 2026. developer.nvidia.com/...healthcare
- ^"Linux Foundation Announces Contribution of Newton by Disney Research, Google DeepMind and NVIDIA to Accelerate Open Robot Learning", The Linux Foundation, 29 September 2025. linuxfoundation.org/...elerate-open-robot-learning
- ^"NVIDIA/warp", repository README, GitHub, accessed 31 July 2026. github.com/...warp
- ^"Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions; Guidance for Industry and Food and Drug Administration Staff; Availability", Federal Register, US Food and Drug Administration, 17 November 2023 (Docket FDA-2021-D-0980). federalregister.gov/...-medical-device-submissions
- ^"Medical Device Development Tools (MDDT)", US Food and Drug Administration, accessed 31 July 2026. fda.gov/...medical-device-development-tools-mddt
- ^"ASME V&V 40-2018: Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices", ANSI Webstore, accessed 31 July 2026. webstore.ansi.org/...asme402018
- ^Sean Whooley, "Nvidia unveils open-source simulation framework for surgical robotics", MassDevice, 22 July 2026. massdevice.com/...tion-framework-surgical-robotics
- ^Ana Maria Constantin, "Nvidia's open-source simulator trains surgical robots in under two minutes", TNW, 23 July 2026. thenextweb.com/...ysics-simulation-surgical-robots
- ^"NVIDIA Launches Open-Source Medical Physics Simulation Framework Within Isaac for Healthcare", HIT Consultant, 22 July 2026. hitconsultant.net/...-physics-simulation-framework
- ^"NVIDIA Cuts Surgical Robot Training From Hours to Minutes With Open-Source Simulator", TechTimes, 23 July 2026. techtimes.com/...urs-minutes-open-source-simulator
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.
v1 · 3,544 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify