Isaac ROS DNN Inference
September 22, 2026 · View on GitHub
NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU.
Webinar Available
Learn how to use this package by watching our on-demand webinar: Accelerate YOLOv5 and Custom AI Models in ROS with NVIDIA Isaac
Overview
Isaac ROS DNN Inference contains ROS 2 packages for performing DNN inference, providing AI-based perception for robotics applications. DNN inference uses a pre-trained DNN model to ingest an input Tensor and output a prediction to an output Tensor.
Above is a typical graph of nodes for DNN inference on image data. The input image is resized to match the input resolution of the DNN; the image resolution may be reduced to improve DNN inference performance, which typically scales directly with the number of pixels in the image. DNN inference requires input Tensors, so a DNN encoder node is used to convert from an input image to Tensors, including any data pre-processing that is required for the DNN model. Once DNN inference is performed, the DNN decoder node is used to convert the output Tensors to results that can be used by the application.
TensorRT and Triton are two separate ROS nodes to perform DNN inference. The TensorRT node uses TensorRT to provide high-performance deep learning inference. TensorRT optimizes the DNN model for inference on the target hardware, including Jetson and discrete GPUs. It also supports specific operations that are commonly used by DNN models. For newer or bespoke DNN models, TensorRT may not support inference on the model. For these models, use the Triton node.
The Triton node uses the Triton Inference Server, which provides a compatible frontend supporting a combination of different inference backends (e.g. ONNX Runtime, TensorRT Engine Plan, TensorFlow, PyTorch). In-house benchmark results measure little difference between using TensorRT directly or configuring Triton to use TensorRT as a backend.
Some DNN models may require custom DNN encoders to convert the input data to the Tensor format needed for the model, and custom DNN decoders to convert from output Tensors into results that can be used in the application. Leverage the DNN encoder and DNN decoder nodes for image bounding box detection and image segmentation, or your own custom nodes.
Note
DNN inference can be performed on different types of input data, including audio, video, text, and various sensor data, such as LIDAR, camera, and RADAR. This package provides implementations for DNN encode and DNN decode functions for images, which are commonly used for perception in robotics. The DNNs operate on Tensors for their input, output, and internal transformations, so the input image needs to be converted to a Tensor for DNN inferencing.
ROS 2 Native rosidl::Buffer Acceleration
This package uses rosidl::Buffer, a feature built into ROS 2 Lyrical, to
avoid unnecessary copies of large payloads between CPU and accelerator
memory. The CUDA buffer backend builds on this native ROS 2 feature to provide
CUDA memory storage and transport. Most applications can use standard ROS
messages and conversion packages without depending directly on a buffer
backend. See rosidl::Buffer and Buffer Backends for details.
Performance
| Sample Graph | Input Size | AGX Thor T5000 | AGX Thor T4000 | AGX Orin | Orin Nano Super 8GB | DGX Spark | x86_64 w/ RTX 5090 | x86_64 w/ RTX 5070 |
|---|---|---|---|---|---|---|---|---|
| TensorRT Node DOPE | VGA | 193 fps 1.4 ms @ 30Hz | 128 fps 1.3 ms @ 30Hz | 47.4 fps 1.9 ms @ 30Hz | 20.8 fps | 108 fps 1.2 ms @ 30Hz | 350 fps 0.53 ms @ 30Hz | 161 fps 0.48 ms @ 30Hz |
| Triton Node DOPE | VGA | 174 fps 6.2 ms @ 30Hz | 116 fps 10 ms @ 30Hz | 45.2 fps 22 ms @ 30Hz | 20.2 fps | 98.1 fps 8.6 ms @ 30Hz | 350 fps 2.9 ms @ 30Hz | 156 fps 6.4 ms @ 30Hz |
| TensorRT Node PeopleSemSegNet | 544p | 792 fps 1.2 ms @ 30Hz | 524 fps 1.4 ms @ 30Hz | 481 fps 1.5 ms @ 30Hz | 234 fps 1.8 ms @ 30Hz | 1030 fps 0.87 ms @ 30Hz | 3000 fps 0.46 ms @ 30Hz | 2320 fps 0.52 ms @ 30Hz |
| Triton Node PeopleSemSegNet | 544p | 195 fps 6.3 ms @ 30Hz | 191 fps 6.7 ms @ 30Hz | 189 fps 5.5 ms @ 30Hz | 122 fps 8.4 ms @ 30Hz | 173 fps 5.7 ms @ 30Hz | 2440 fps 0.53 ms @ 30Hz | 1630 fps 0.74 ms @ 30Hz |
| DNN Image Encoder Node | 1200p | 767 fps 2.5 ms @ 30Hz | 418 fps 4.6 ms @ 30Hz | 423 fps 4.5 ms @ 30Hz | 475 fps 4.5 ms @ 30Hz | 1130 fps 1.9 ms @ 30Hz | 2870 fps 0.91 ms @ 30Hz | 449 fps 2.5 ms @ 30Hz |
Documentation
Please visit the Isaac ROS Documentation to learn how to use this repository.
Packages
Latest
Update 2026-09-21: Migrated the TensorRT and Triton nodes from NITROS to rosidl::Buffer with the CUDA buffer backend