NVIDIA Holoscan
Last reviewed
Sources
11 citations
Review status
Source-backed
Revision
v1 · 1,801 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
11 citations
Review status
Source-backed
Revision
v1 · 1,801 words
Add missing citations, update stale details, or suggest a clearer explanation.
NVIDIA Holoscan is a domain-agnostic, multimodal AI sensor processing platform and software development kit (SDK) built by NVIDIA for real-time, low-latency processing of streaming sensor data combined with AI inference at the edge. Holoscan provides a full-stack runtime that ingests high-bandwidth data from sensors such as cameras, lidar, and radar, moves it directly into GPU memory, runs accelerated computer vision and deep learning pipelines on it, and produces deterministic real-time outputs. Originally created for software-defined medical devices under the Clara Holoscan brand, the platform has since broadened into a general runtime for physical AI and sensor processing across healthcare, scientific instruments, industrial systems, robotics, and live media.[1][2]
Holoscan is closely tied to NVIDIA's edge and embedded compute lines, including the IGX industrial platform and the Jetson family. In June 2026, the Holoscan Sensor Bridge, the platform's real-time sensor connectivity layer, was positioned as a core component of NVIDIA Halos for Robotics, extending the functional safety chain from compute all the way out to the sensor edge.[3][4]
NVIDIA describes Holoscan as "a domain-agnostic, multimodal AI sensor processing platform" that delivers the accelerated, full-stack infrastructure needed for real-time processing of streaming data.[1] In practice it is a runtime and SDK that lets developers build applications which acquire raw sensor data, process and analyze it with GPU acceleration (including AI inference), and act on the results with low and predictable latency. NVIDIA frames the platform more recently as "a runtime for physical AI and sensor processing," emphasizing GPU-accelerated systems that handle high-bandwidth sensor input, perform AI reasoning, and execute real-world actions deterministically.[2]
Key characteristics include a graph-based execution engine, accelerated input/output that writes data directly into GPU memory, and the ability to scale to tens or hundreds of gigabits per second of sensor throughput without extensive system optimization. The platform spans NVIDIA's hardware range, from embedded Jetson modules to enterprise DGX systems.[2]
Holoscan began life inside NVIDIA's Clara healthcare portfolio. NVIDIA Clara was first introduced in 2018 as a medical imaging and computing initiative, and at GTC in November 2021 the company announced Clara Holoscan, a computing platform that lets developers build software-defined medical devices running low-latency streaming applications at the edge. The early Clara Holoscan systems were powered by the NVIDIA Jetson AGX Orin module.[5]
At GTC in March 2022, NVIDIA launched Clara Holoscan MGX, a hardware and software stack described as an open, scalable platform for connecting robotic and AI-enabled medical devices and sensors to real-time AI applications. The goal was to bring high-performance AI inference directly into clinical and surgical settings, where determinism, reliability, and regulatory compliance are essential.[6][7]
As the SDK matured, NVIDIA generalized it. Beginning with Holoscan SDK version 0.4.0, Holoscan was officially repositioned as a domain-agnostic platform usable for sensor AI applications across many fields, and the "Clara" prefix was dropped from the general SDK while Clara remained NVIDIA's healthcare brand.[1][6]
The Holoscan SDK is the developer-facing core of the platform. Applications are expressed as computation graphs, or pipelines, built from modular processing components called operators. Each operator performs a unit of work, such as capturing a frame, running an inference model, performing image processing, or rendering visualization, and operators are connected into a directed graph through which sensor data flows. This operator-and-graph model lets developers compose complex real-time pipelines from reusable building blocks.[1]
Underneath, Holoscan applications execute on the Graph Execution Framework (GXF), NVIDIA's underlying scheduling and execution engine for graph-based, accelerated workloads. The SDK is available in both C++ and Python, and NVIDIA distributes it as an open-source project on GitHub, alongside a curated collection of reference applications and reusable operators through the HoloHub community repository.[1]
The SDK supports a range of compute platforms and architectures:
| Aspect | Details |
|---|---|
| Compute platforms | NVIDIA IGX (industrial edge), Jetson (embedded edge), x86 workstations and servers |
| Architectures | x86_64 and aarch64 (ARM) |
| Acceleration | NVIDIA GPUs; accelerated IO directly into GPU memory |
| Languages | C++ and Python APIs |
| Distribution | PyPI, Debian packages, NGC containers, Conda, open source on GitHub |
Reference applications shipped or showcased with the SDK span medical imaging and surgical video, computer vision tasks such as body pose estimation and object segmentation, speech-to-text and large language model integration, and extended reality visualization.[1] NVIDIA continues to extend the SDK with releases such as Holoscan 3.0, which added dynamic flow control for building more flexible edge AI applications.[8]
The Holoscan Sensor Bridge (HSB) is the platform's real-time sensor connectivity layer. It is an FPGA-based, sensor-over-Ethernet interface that ingests high-bandwidth data from peripheral devices such as cameras, lidar, and radar and delivers it to the host system for GPU processing. NVIDIA promotes HSB under a "Bring Your Own Sensor" (BYOS) approach, in which a wide variety of sensors can be connected over standard Ethernet.[9][10]
The data flow works as follows: peripheral sensor data is acquired by the Holoscan Sensor Bridge device's FPGA and transmitted as UDP over Ethernet to the host. On systems equipped with NVIDIA ConnectX SmartNICs, such as the IGX developer kit or DGX Spark, the ConnectX interface uses GPU RDMA (remote direct memory access) to write the incoming UDP data directly into GPU memory, bypassing the CPU and minimizing latency. Systems without an accelerated NIC, such as some Jetson platforms, fall back to standard Linux socket-based Ethernet, trading throughput for broader hardware compatibility. The Holoscan SDK then receives and processes the streams on the GPU.[10]
Reported characteristics of the Holoscan Sensor Bridge include:
| Property | Detail |
|---|---|
| Interface | FPGA-based, sensor-over-Ethernet ingest |
| Transport | UDP over Ethernet, with GPU RDMA via ConnectX SmartNICs |
| Memory path | Direct write of sensor data into GPU memory |
| Data rates | Scalable from roughly 100 Mbps to over 100 Gbps |
| Latency | End-to-end latencies reported as low as about 17 ms |
| Synchronization | High-accuracy PTP (Precision Time Protocol) timestamps on IGX and Jetson |
| Sensors | Cameras, lidar, radar; examples include Sony IMX274 and Leopard Imaging multi-camera modules over CSI-2 |
| Compute targets | IGX devkit, DGX Spark, Jetson AGX Orin, Jetson AGX Thor |
By moving sensor data into GPU memory with minimal CPU overhead and a unified API, HSB targets low-latency sensor ingest for physical AI, robotics, medical imaging, and industrial applications.[9][10]
While Holoscan started in medical devices, it now serves several domains:
On June 22, 2026, NVIDIA announced Halos for Robotics, described as the industry's first full-stack, comprehensive safety system for robotics and physical AI, unifying AI compute and safety. Halos extends safety methodology from autonomous vehicles to physical AI systems operating in industrial environments.[3]
Within Halos, the Holoscan Sensor Bridge provides the real-time sensor connectivity layer. It connects sensors and actuators to NVIDIA IGX Thor over Ethernet, extending the safety chain all the way to the sensor edge. NVIDIA highlights several HSB capabilities in this context: low latency via ConnectX RDMA and RTX GPU Direct for real-time streaming; scalability to hundreds of sensors and gigabit-per-second data rates; multimodal support for any sensor or actuator type; and security through MACsec authentication together with an end-to-end IEC 61508 SIL 2 safety protocol. The HSB data path runs from camera through FPGA IP modules for sensor control, packetization, watermarking, and MACsec, before reaching the host GPU for synchronized multimodal perception in safety-critical workloads.[4]
The compute foundation for this configuration is NVIDIA IGX Thor, which NVIDIA states delivers up to 2,070 FP4 TFLOPS of AI performance, 14 Neoverse ARM CPU cores, and 128 GB of memory at 273 GB/s bandwidth. IGX Thor adds embedded functional safety, including an IEC 61508 SIL 3 capable Safety Island with independent I/O, power, and clocks, high diagnostic coverage via more than 22,000 safety mechanisms, redundancy and diversity for ASIL decomposition, and in-system test for latent fault coverage. Together, Holoscan Sensor Bridge and IGX Thor allow safety-critical sensor data to flow into GPU-accelerated perception with deterministic, low-latency behavior.[4]
NVIDIA listed a broad ecosystem around Halos, including Agility as a lead adopter and partners such as Acontis, FreeRTOS, QNX, Advantech, NexCobot, Infineon, NXP, STMicroelectronics, Texas Instruments, FORT Robotics, KION Group, Lyte AI, and Neurealm, along with certification bodies including TUV Rheinland, TUV SUD, UL Solutions, exida, SGS, and CertX.[3]
Holoscan sits at the intersection of NVIDIA's edge AI hardware and its physical AI software. The platform runs on the IGX industrial edge platform, which provides functional safety and security, and on Jetson embedded modules for smaller form factors, while also supporting x86 servers and DGX systems for development and high-throughput workloads.[1][2] Accelerated IO is delivered through NVIDIA networking technologies such as Rivermax and the Holoscan Sensor Bridge, both of which move data into GPU memory with minimal CPU involvement.[2] Through these connections, Holoscan serves as a real-time sensor processing runtime within NVIDIA's broader physical AI and robotics strategy.