NVIDIA NIM

NVIDIA NIM (NVIDIA Inference Microservices) is a set of containerized, prebuilt-and-optimized model-serving microservices from NVIDIA that package an AI model, an optimized inference engine, and an OpenAI-compatible REST API into a single Docker container for deployment across cloud environments, data centers, workstations, and edge hardware. NVIDIA announced NIM at GTC on March 18, 2024, and built it on its own inference software, including NVIDIA Triton Inference Server and NVIDIA TensorRT-LLM, with NVIDIA stating the prebuilt containers "enable developers to reduce deployment times from weeks to minutes."^[1] Enterprises can pull a NIM from the NGC container registry, run it on any NVIDIA GPU infrastructure, and begin serving requests within minutes rather than weeks.^[1][3]

NIM is distributed as part of NVIDIA AI Enterprise and is optimized to run across what NVIDIA describes as its "CUDA installed base of hundreds of millions of GPUs across clouds, data centers, workstations and PCs."^[1] By tying optimized inference to NVIDIA's GPU lineup and offering production access through an NVIDIA AI Enterprise license, NVIDIA positions NIM as both a developer convenience layer and an enterprise software subscription. The software underneath has changed since launch: the NIM LLM and VLM 2.0 containers documented in 2026 run one inference engine per container image, vLLM for the standard LLM and VLM image (NVIDIA's migration guide calls vLLM "the sole inference backend"), with SGLang-variant images published for a few models, and NVIDIA's own documentation now describes NIM as "an enterprise orchestration layer for vLLM" rather than a bundle of interchangeable engines.^[29][30][31]

Jensen Huang introduced NIM during his GTC 2024 keynote at the San Jose Convention Center, framing it as a way to turn NVIDIA's GPU installed base into a production AI deployment platform. "Established enterprise platforms are sitting on a goldmine of data that can be transformed into generative AI copilots," Huang said in the launch announcement.^[1] The announcement came alongside a set of healthcare-specific NIMs under the BioNeMo brand, covering drug discovery, medical imaging, and genomics.^[10] NVIDIA described NIM as part of its broader strategy to sell software and services on top of its GPU hardware business, with NVIDIA AI Enterprise providing the commercial licensing framework.

What problem does NIM solve?

Before NIM, deploying an open-weight language model to production required assembling several moving parts manually. A team would download model weights from a repository such as Hugging Face, select and configure an inference server (such as vLLM, Text Generation Inference, or NVIDIA Triton Inference Server), compile TensorRT engines for the target GPU architecture, write a serving wrapper that exposed an HTTP endpoint, and then containerize the whole stack. For a team without deep MLOps experience, that process could consume weeks of engineering time. For teams who did have that experience, the work was repeatable but not portable: an engine compiled for an A100 cluster would not run on an H100 without recompilation.

The problem was worse at scale. Running multiple models simultaneously meant maintaining separate container images, separate version pins, and separate optimization profiles. Updating a model from one version to the next triggered a full revalidation cycle. Fine-tuned adapters added another layer of bookkeeping.

NVIDIA's response was to pre-solve all of that for its own GPU ecosystem. Each NIM ships with a manifest of pre-validated profiles, and the container selects one at launch based on the detected hardware.^[31] In the original 1.x containers the operator did not need to know whether TensorRT-LLM or vLLM was running under the hood, because both were bundled and the container chose between them; the 2.0 generation replaced that multi-backend design with a single backend per container image, vLLM in the standard image (see below).^[29][31]

The timing of NIM's release reflected industry dynamics in early 2024. The open-weight model ecosystem had matured significantly: Meta's Llama 2 and Mistral 7B had demonstrated that capable models could be self-hosted, and demand for on-premises inference was growing among enterprises that could not or would not send proprietary data to third-party API providers. At the same time, the tooling to actually deploy those models remained fragmented and difficult to validate. NIM addressed that gap by treating inference deployment as a product rather than a DIY engineering problem.

How is a NIM container built?

Container structure

A NIM container is a Docker image stored in NVIDIA's NGC container registry at nvcr.io. The original (1.x) LLM images bundled several backends in one container: the base model weights (or a pointer to fetch them from NGC at runtime), one or more compiled TensorRT-LLM engine profiles tuned for specific GPU architectures, a vLLM or SGLang runtime for GPUs without a compiled TensorRT-LLM profile, an OpenAI-compatible HTTP server, CUDA libraries, and an enterprise base layer with security patches.^[29]

NIM LLM and VLM 2.0 replaced that design with what NVIDIA calls a "one container, one backend" architecture.^[3] According to the 2.0 documentation, the container runs two supervised processes: a vLLM inference backend that loads the model, runs GPU inference and exposes an OpenAI-compatible API on an internal loopback port (8001), and a thin nginx proxy on the external port (8000 by default, overridable with NIM_SERVER_PORT) that provides liveness and readiness probes (/v1/health/live, /v1/health/ready), request routing, TLS termination and CORS handling.^[30] Only explicitly configured endpoints are exposed; every other path returns 404, and if either process exits the container shuts down so an orchestrator can reschedule it.^[30] On top of these sit two NVIDIA layers: nim-llm, the orchestration entry point that merges configuration from CLI flags, environment variables and runtime configs and injects features such as LoRA adapters, and nimlib, which handles model licensing, hardware-aware profile selection, model download and the management API.^[3]

The 2.0 documentation also distinguishes model-specific NIMs (a curated manifest, curated weights and validated quantization profiles for one model) from model-free NIMs, which generate their manifest at runtime and can serve a model pulled from NGC, Hugging Face, Amazon S3, Google Cloud Storage or a local directory. NVIDIA positions model-free NIMs for newly released architectures that vLLM supports before an official NIM exists, for custom fine-tuned models, and for security teams that would rather approve one container than one per model.^[3]

To pull and start a model-specific NIM, an operator typically authenticates to the NGC registry with an API key, sets a local cache directory, and runs a single docker run command. NVIDIA's Nemotron 3 Ultra walkthrough uses NGC_API_KEY for the registry login (docker login nvcr.io --username '$oauthtoken'), LOCAL_NIM_CACHE (mounted at /opt/nim/.cache) for downloaded weights, and a pinned image tag such as nvcr.io/nim/nvidia/nemotron-3-ultra-550b-a55b:2.0.12.^[28] The 2.0 documentation states that eligible public-catalog NIMs can be pulled and run without an NGC API key.^[3]

Inference engine selection

NIM uses a profile system to decide how a model is served. Every model-specific NIM ships with a model manifest, a catalog of one or more profiles, each a pre-validated configuration defined by tags for the backend engine, model precision, tensor-parallelism size, pipeline-parallelism size and LoRA support. At startup the container selects exactly one profile, and that choice determines which model files are downloaded and how the backend is launched.^[31]

The documented naming convention for 2.0 profiles is <backend>-<precision>-tp<N>-pp1[-lora], where the backend tag is vllm, sglang or trtllm and must match the engine in the container image being run (an SGLang image only loads sglang-* profiles), the precision tag is one of bf16, fp8, mxfp4 or nvfp4, tp<N> is the tensor-parallelism degree in GPUs, and pp1 denotes single-stage pipeline parallelism.^[31] The profile string NVIDIA's September 2026 blog post tells operators to set for Nemotron 3 Ultra on four B200 GPUs is vllm-nvidia-b200-nvfp4-tp4-pp1-throughput-90.0 (the support matrix for the same NIM lists its validated profiles in the shorter form, for example vllm-nvfp4-tp4-pp1), which names the vLLM backend, the B200 target, NVFP4 precision, four-way tensor parallelism with no pipeline parallelism, and a throughput objective.^[28]

Running the image with the list-model-profiles command prints every profile in the manifest with its 64-character profile ID, its human-readable description and an estimated per-GPU VRAM requirement, grouped into profiles that are compatible with the host, compatible but short of memory at full context length (with a suggested --max-model-len reduction), and incompatible.^[31] Selection follows a priority chain: NIM_MODEL_PROFILE="default" picks the first hardware-compatible profile using backend priority; NIM_MODEL_PROFILE=<id-or-description> selects an explicit profile by checksum or friendly name; otherwise a memory-aware selector estimates the VRAM needed for weights, KV cache, activations and overhead and filters out profiles that do not fit, before the manifest's own selection criteria choose among the remainder.^[31] Backend CLI arguments passed to the container (for example vLLM's --tensor-parallel-size) override the profile's settings, and NIM resolves those overrides before downloading model files so the artifacts match the final launch configuration.^[31]

export NIM_MODEL_PROFILE=vllm-nvidia-b200-nvfp4-tp4-pp1-throughput-90.0
docker run --gpus all --shm-size=16GB -e NGC_API_KEY -e NIM_MODEL_PROFILE \
  -e NIM_SPECDEC_ENABLE=1 -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" -p 8000:8000 \
  nvcr.io/nim/nvidia/nemotron-3-ultra-550b-a55b:2.0.12

The example above is condensed from NVIDIA's Nemotron 3 Ultra deployment guide; NIM_SPECDEC_ENABLE=1 turns on speculative decoding for that model.^[28]

The TensorRT-LLM-first selection with a vLLM fallback was the 1.x behavior. In the 2.0 generation the backend is fixed per container image, the standard LLM and VLM container is built on vLLM, and NVIDIA's flagship throughput profile for Nemotron 3 Ultra on Blackwell is a vLLM profile.^[3][28][29]

TensorRT-LLM backend

In the 1.x containers, NVIDIA positioned the TensorRT-LLM profile as the highest-throughput option among the bundled backends. TensorRT-LLM compiles the model graph into a GPU-specific execution plan, fuses operations, and applies continuous batching along with paged KV-cache management.^[14] NVIDIA published two TensorRT-LLM profile variants per model: a latency profile that minimizes time-to-first-token and inter-token latency for interactive workloads, and a throughput profile that maximizes tokens generated per second for batch workloads.

On H100 SXM hardware, NVIDIA's 2024 benchmarks showed approximately 2x throughput improvement for Llama 3.1 8B compared to a default deployment: roughly 1,200 tokens per second with TensorRT-LLM optimization versus around 610 tokens per second without it.^[2]

NIM LLM and VLM 2.0 dropped the multi-backend container. NVIDIA's migration guide states that 2.0 "uses a new architecture built on vLLM as the sole inference backend",^[29] and the 2.0 overview says "the multi-backend container (vLLM, TensorRT-LLM, and others) is replaced by a dedicated vLLM container."^[3] The 2.0 overview lists "direct access to raw vLLM capabilities without the latency of abstraction layers" as a benefit, says enterprise shims have been streamlined so that features are driven upstream rather than maintained as downstream forks, and describes NVIDIA as contributing optimizations to the vLLM project.^[3] The migration guide also notes that the nim-run command was replaced by nim-serve, that the nvext request extension object was removed in favor of top-level vLLM sampling parameters, that NIM_MODEL_NAME became NIM_MODEL_PATH, and that many 1.x environment variables were replaced by native vLLM engine arguments.^[29] TensorRT-LLM itself remains a separate NVIDIA project and is one of the backends supported by NVIDIA Dynamo.^[17]

OpenAI-compatible API layer

The HTTP server inside each NIM exposes endpoints that are structurally identical to OpenAI's API.^[3] A developer who has code calling client.chat.completions.create() from the OpenAI Python SDK can point that code at a locally running NIM by changing a single base URL, with no other modifications. The 1.x containers also exposed NVIDIA-specific extensions for features that had no OpenAI equivalent, such as per-request LoRA adapter selection; the 2.0 migration guide records that the nvext request extension was removed and that sampling parameters now live in top-level request fields matching vLLM's API.^[29] In 2.0 the proxy routes /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models and /v1/metrics to the vLLM backend and rejects every other path with 404.^[30]

The /v1/models endpoint returns metadata about the loaded model, and the OpenAI-compatible format allows direct integration with frameworks such as LangChain, LlamaIndex, and most other LLM orchestration libraries.

PEFT and LoRA adapter support

NIM supports dynamic loading of Low-Rank Adaptation (LoRA) adapters without restarting the container. Adapters in Hugging Face PEFT format can be placed in a directory named by NIM_PEFT_SOURCE; in 2.0 they are loaded at startup (static LoRA) or picked up while the server runs through directory polling (NIM_PEFT_REFRESH_INTERVAL) or a runtime API (dynamic LoRA).^[7] The 2.0 migration guide states that NeMo-format adapters are no longer supported and must be converted to Hugging Face PEFT format.^[29] When a client submits an inference request, it can specify a named LoRA adapter; the server pulls the corresponding weights and applies them to the base model computation.

This multi-LoRA serving mode lets a single NIM container serve dozens of fine-tuned variants of one base model simultaneously, without duplicating the base weights on disk or in GPU memory. NVIDIA's documentation refers to deploying a large collection of adapters this way as a "swarm of LoRA adapters."^[8]

Kubernetes and orchestration

NVIDIA ships Helm charts for deploying NIM on Kubernetes clusters. The NVIDIA NIM Operator (available as a separate component) automates the lifecycle of NIM pods, handling version upgrades, health checks, and GPU resource scheduling. NIM integrates with NVIDIA's GPU Operator for Kubernetes, which manages CUDA drivers and device plugins across cluster nodes.

For multi-NIM deployments, NVIDIA provides the NVIDIA Inference Manager (part of NVIDIA AI Enterprise), which routes requests across multiple running NIMs and provides load balancing.

NIM Operator

The NVIDIA NIM Operator is a Kubernetes operator designed to automate NIM deployment and management within a cluster. It handles the full pod lifecycle: pulling updated container images from NGC, draining connections before rolling restarts, restoring previous versions on failure, and reporting health status to Kubernetes cluster management tools. The operator reads a custom resource definition (CRD) that specifies which NIM to run, the GPU resource limits, and the model cache mount path. Once the CRD is applied, the operator handles the rest, including authentication to the NGC registry using a Kubernetes secret.

For air-gapped deployments (environments with no internet access), NVIDIA supports pre-pulling NIM container images to an internal container registry and mounting pre-downloaded model weights from a local network storage volume, so that NIMs never need to contact NVIDIA's infrastructure after initial setup.

Observability and monitoring

NIM LLM and VLM 2.0 containers expose Prometheus-compatible metrics at a /v1/metrics endpoint, alongside the /v1/health/live and /v1/health/ready probes served by the proxy; the migration guide directs 1.x users to re-point metric scraping at the metrics vLLM exposes.^[29][30] Standard metrics include request count, request latency histograms, token throughput (input tokens per second and output tokens per second), batch size distribution, KV-cache utilization, and GPU memory usage. These metrics integrate with Grafana dashboards for production monitoring. NVIDIA provides reference dashboard configurations alongside its Helm charts.

Request-level logging can be configured to emit structured JSON Lines logs (NIM_LOG_LEVEL and NIM_JSONL_LOGGING in 2.0), with X-Request-Id and Traceparent headers forwarded for distributed tracing,^[29][30] which can be forwarded to centralized logging systems such as Elasticsearch or Splunk. Log fields include request ID, model name, prompt token count, completion token count, latency, and the selected inference profile.

NVIDIA Dynamo and distributed serving

At GTC 2025 on March 18, 2025, NVIDIA introduced NVIDIA Dynamo, an open-source distributed inference framework aimed at serving large reasoning models across many GPUs in what NVIDIA calls an AI factory.^[16] Dynamo builds on the modular architecture of NVIDIA Triton Inference Server and supports multiple backends, including PyTorch, SGLang, TensorRT-LLM, and vLLM.^[17] Its core innovations are disaggregated serving (running the compute-bound prefill phase and the memory-bound decode phase on separate GPUs), a smart router that tracks the KV cache across a GPU fleet to avoid redundant recomputation, a GPU planner that adds and removes GPUs as demand shifts, and a distributed KV cache manager that offloads less-frequently-accessed cache to CPU memory, local storage, or networked object storage.^[16][17] NVIDIA reported that Dynamo boosts throughput on the DeepSeek-R1 671B model running on a Blackwell GB200 NVL72 system by 30x, and more than doubles performance on Llama 70B running on Hopper GPUs.^[16] For enterprises, NVIDIA stated that Dynamo would be included with NVIDIA NIM microservices as part of NVIDIA AI Enterprise, and the framework is published openly in the ai-dynamo/dynamo GitHub repository.^[16][17] That integration shipped in August 2026 as NIM LLM 3.0.0, a series NVIDIA runs alongside 2.0 rather than in place of it: the same container can start in standard mode (the single-container 2.0 behavior) or in an opt-in Dynamo mode that separates an OpenAI-compatible frontend and KV-cache-aware router from independently scaled vLLM workers. NVIDIA's release notes say 2.0 remains the default image on NGC, that 3.0.0 is not NIM Certified, and that validated Dynamo recipes are published for nemotron-3-super-120b-a12b and nemotron-3-ultra-550b-a55b.^[37]

What are NIM Agent Blueprints?

In August 2024, NVIDIA extended the NIM platform with NIM Agent Blueprints, a catalog of pre-built, end-to-end generative AI application workflows.^[4] A Blueprint packages multiple NIM containers, orchestration logic, sample code, and deployment charts for a specific enterprise use case. Developers can take a Blueprint, plug in their own data and branding, and deploy a functioning application rather than building each component from scratch.^[5]

The initial Blueprint releases covered three use cases:

• A digital human workflow for customer service, creating a real-time 3D animated avatar interface driven by a speech recognition NIM, an LLM NIM, and a text-to-speech NIM
• A generative virtual screening workflow for drug discovery, combining molecular generation models with protein structure prediction to accelerate candidate identification
• A multimodal PDF data extraction workflow for enterprise retrieval-augmented generation (RAG), using NVIDIA NeMo Retriever NIMs to process PDF documents and build high-accuracy retrieval pipelines

NVIDIA announced plans for monthly Blueprint releases covering customer service, content generation, software engineering, retail shopping advisors, and R&D applications.^[4] Partners including Accenture, Deloitte, Cisco, Dell Technologies, Hewlett Packard Enterprise, Lenovo, SoftServe, and World Wide Technology were among the first systems integrators offering Blueprint-based solutions to enterprise customers.^[4]

Blueprints are published on build.nvidia.com alongside documentation, sample code, and Helm charts. Each Blueprint is designed around what NVIDIA calls a "data-driven generative AI flywheel": user interactions with the deployed application generate new data, which can be used to fine-tune LoRA adapters, which are then loaded back into the NIM without restarting the service.^[5] NVIDIA NeMo handles the fine-tuning step, and NVIDIA AI Foundry provides the production environment for managing adapter versions.

The customer service digital human Blueprint integrates several NIM types in sequence. An audio NIM transcribes the user's voice. An LLM NIM generates a text response. A text-to-speech NIM converts the text to audio. An Avatar NIM (from NVIDIA ACE, the Avatar Cloud Engine platform) drives the facial animation of a rendered character. All four components communicate over the local network using the same OpenAI-compatible API format, which makes it possible to swap any component for an alternative NIM without rewriting the orchestration logic.

NVIDIA later rebranded NIM Agent Blueprints simply as NVIDIA Blueprints, expanding the catalog beyond agentic workflows to include reference architectures for video analytics, document processing, and scientific computing.

At GTC 2025, NVIDIA added the AI-Q Blueprint, a reference workflow for connecting enterprise knowledge sources to teams of autonomous AI agents, which it scheduled for availability in April 2025.^[18] The blueprint did ship: a NVIDIA technical blog post dated 11 June 2025 described it as available on GitHub and on build.nvidia.com, packaging an AI-Q Research Assistant example on top of NIM, NeMo Retriever, and the NeMo Agent toolkit.^[27] Alongside it, NVIDIA released an open-source agent toolkit (the Agent Intelligence toolkit, later known as NeMo Agent toolkit) on GitHub for orchestrating multi-agent systems built on NIM and NeMo microservices.^[18]

What models does NIM support?

The NIM catalog at build.nvidia.com lists over 100 models across multiple domains.^[3] The catalog is organized into several categories.

Large language models

The LLM catalog includes models from multiple providers packaged as NIMs:

Llama 3.1 405B, one of the largest openly licensed models available, requires multiple H100 GPUs and is served with tensor parallelism enabled across those GPUs.

NVIDIA also packages its own fine-tuned Nemotron models. Llama 3.1 Nemotron 70B is a Nemotron post-trained version of Llama 3.1 70B that NVIDIA released as an instruction-following model with particularly high scores on MT-Bench. The Nemotron-Mini 4B model is tuned for edge and workstation deployment on RTX hardware.

The model catalog is updated regularly as new open-weight models are released. NVIDIA typically adds a NIM for a major new model within weeks of the model's public release; under the 2.0 offerings the fast-track NIM tier is published within about 72 hours of upstream availability, with broader hardware validation following in NIM Certified.^[32] Several high-profile 2025 frontier and open models were packaged as NIM microservices, including Meta's Llama 4 Scout and Llama 4 Maverick (released April 5, 2025) and OpenAI's gpt-oss-120b and gpt-oss-20b open-weight models (released August 5, 2025).^[19][20]

Embedding models

NIM includes a set of text and multimodal embedding NIMs for search and retrieval applications:

Vision language models

Vision language model NIMs accept image plus text inputs and generate text outputs:

Speech models

NVIDIA's Riva speech platform is packaged as NIM containers for automatic speech recognition (ASR), text-to-speech synthesis (TTS), neural machine translation (NMT), and speaker diarization. Riva NIMs include both streaming and batch inference modes, with the streaming ASR NIM able to return partial transcriptions with low latency, suitable for live transcription applications.

Specialized domain models

For healthcare and life sciences, NVIDIA offers BioNeMo NIMs covering protein structure prediction (ESMFold), generative molecular design (MolMIM), molecular docking (DiffDock), and genomics analysis. These launched alongside the initial NIM announcement at GTC 2024, with NVIDIA describing them as more than two dozen healthcare-specific microservices.^[10]

ESMFold is a protein language model from Meta that predicts protein 3D structure from amino acid sequences. Packaged as a NIM, it exposes a REST API that accepts a FASTA-format amino acid string and returns structure predictions in PDB format, allowing downstream molecular visualization and docking tools to consume the output directly.

MolMIM (Molecular Multimodal Information Model) is a generative chemistry model that proposes new drug-like molecules with specified physicochemical properties. The MolMIM NIM accepts a SMILES string and a set of property constraints, and returns novel candidate molecules ranked by predicted property values.

DiffDock is a diffusion model for protein-ligand docking. Given a protein structure and a small molecule, it predicts the 3D binding pose, which is useful for evaluating whether a candidate compound is likely to bind to a therapeutic target.

Visual generative AI models

NIM also covers image and video generation, including Stable Diffusion and other diffusion-based models, available through the visual generative AI NIM catalog. These NIMs accept text prompts (and optionally input images) and return generated images via a REST API. Enterprise deployments use these for content creation pipelines, product visualization, and synthetic data generation for training computer vision models.

OpenUSD and 3D world models

At SIGGRAPH in July 2024, NVIDIA introduced a set of NIM microservices built around OpenUSD (Universal Scene Description) to accelerate the development of digital twins, robotics simulation, and industrial 3D workflows that run in NVIDIA Omniverse.^[21] These include the USD Code NIM (which generates OpenUSD Python code from text prompts), the USD Search NIM (natural-language and image search across 3D asset libraries), and the USD Validate NIM (which checks file compatibility and renders RTX preview images), along with additional services for scene layout, material application, mesh generation from point-cloud data, physics super-resolution, and large-scale neural radiance fields.^[21] Foxconn and WPP were named among the first adopters, using the microservices for factory digital twins and content generation pipelines respectively.^[21]

Media and broadcast models

NVIDIA's AI for Media stack (formerly NVIDIA Maxine) packages real-time audio and video enhancement models as NIM microservices for media and entertainment workflows. A 2026 product update introduced a lineup of SMPTE ST 2110-compliant NIM microservices designed to drop directly into live, IP-based broadcast pipelines, including LipSync, Active Speaker Detection, Studio Voice (speech enhancement), and Video Super Resolution.^[22] The update also added the NVIDIA Maxine Synthetic Video Detector, a NIM microservice that estimates the probability that a video clip was generated by AI diffusion models using an ensemble of DINOv2 and DINOv3 features with TensorRT optimization.^[23] NVIDIA reported that the detector reaches roughly 92 percent accuracy on uncompressed video and can process frames in as little as 22 milliseconds, and noted that it was offered in private access at the time of the update.^[22][23]

How much does NIM cost?

NIM has two access tiers with distinct terms.

Development access

Any developer with a free NVIDIA Developer Program account can generate an NGC API key and pull NIM containers for development and testing. The hosted inference API at build.nvidia.com provides free access to NIM endpoints (backed by NVIDIA DGX Cloud infrastructure) for evaluation, limited by a rate cap per key. This tier does not require an NVIDIA AI Enterprise license and is intended for prototyping.

Production access

Deploying NIM in production requires an NVIDIA AI Enterprise license.^[6] NVIDIA AI Enterprise is a commercial software subscription that includes NIM along with other NVIDIA AI frameworks such as NeMo, Riva, and RAPIDS.

NVIDIA AI Enterprise list pricing for self-managed systems (as published in the September 2, 2026 revision of NVIDIA's licensing guide):^[6]

The licensing guide notes that the 5-year subscription is included with NVIDIA H100, H200 NVL and A800 PCIe GPUs, and that all subscriptions carry NVIDIA Enterprise Business Standard Support, with 24/7 Business Critical Support and a Technical Account Manager available as upgrades.^[6]

The entry production license is $4,500 per GPU per year, or $1 per GPU per hour for cloud deployments.^[6] Cloud deployments on AWS, Google Cloud, Azure, and Oracle Cloud (selected software components only) can use that pay-as-you-go model, billed on top of the cloud provider's instance costs; the guide also lists a development tier on the same marketplaces (free to use or bring-your-own-license, plus instance costs, with forum and Discord support) and custom-quoted one- to three-year private offers.^[6]

Qualified education and research institutions and members of NVIDIA's Inception and Connect programs pay a quarter of list price. Inception members can purchase up to 64 one-year subscriptions at the reduced price within a 12-month period; Connect program members, who develop publicly available software, can buy up to 16.^[6]

Pricing is per GPU rather than per NIM or per model. A server with eight H100 GPUs running a single NIM costs the same license fee as one running eight NIMs on the same hardware.

NVIDIA AI Enterprise also offers a 90-day trial license for organizations that want to test production NIM deployment before committing to a subscription.^[38]

NIM and NIM Certified

Since the 2.0 documentation, NVIDIA has published inference microservices under two offerings whose terms differ from the simple development-versus-production split above.^[32]

NVIDIA describes NIM Certified as adding broad compatibility across its installed hardware base, a documented refresh cadence, CVE handling, rolling inference-stack updates, OSRB compliance, FedRAMP-ready branches for government use, and validation aligned to the NVIDIA AI Enterprise Feature Branch and Production Branch rules; Government Ready images are offered for designated Production Branch containers.^[3][32] The September 2026 Nemotron 3 Ultra case study summarizes the same offering as "regular inference-stack updates, CVE handling, broader hardware validation, and commercial support through NVIDIA AI Enterprise."^[28] The list of NIM Certified models is maintained on the NGC Catalog.^[32]

NVIDIA AI Enterprise bundles more than just NIM. The subscription includes NVIDIA NeMo (for fine-tuning and training), NVIDIA Riva (for speech AI), NVIDIA Merlin (for recommendation systems), NVIDIA cuOpt (for route optimization), and NVIDIA RAPIDS (for GPU-accelerated data science). This bundling means that organizations already using other NVIDIA AI software may find that NIM production access comes included in a license they already hold.

How is NIM deployed and accessed?

NIM is accessible through three primary routes:

Hosted API (build.nvidia.com): Free endpoints hosted on DGX Cloud hardware, returning completions via the OpenAI-compatible format. Suitable for prototyping and evaluation. Rate-limited per API key.

Self-hosted containers: Pull from nvcr.io using an NGC API key, run on any supported NVIDIA hardware. Requires NVIDIA AI Enterprise for production. Operators have full control over infrastructure, data residency, and model versions.

Cloud marketplace deployments: AWS SageMaker, Google Cloud Vertex AI, Microsoft Azure AI, and Oracle Cloud all offer NIM through their respective AI model catalogs.^[11] On these platforms, the NIM container runs on GPU instances managed by the cloud provider, and the NVIDIA AI Enterprise license can be included in the cloud billing.

NVIDIA also supports deployment on NVIDIA-Certified Systems, a program covering validated server hardware from Cisco, Dell Technologies, Hewlett Packard Enterprise, Lenovo, and Supermicro. For on-premises deployments, validated Kubernetes distributions include Red Hat OpenShift, VMware Tanzu, and Canonical Charmed Kubernetes.

How does NIM compare to vLLM, Triton, and other inference platforms?

The enterprise inference deployment space includes several alternatives to NIM, each with different tradeoffs.

NIM vs. vLLM: vLLM is the closest open-source analog to NIM. Both use continuous batching, paged attention, and OpenAI-compatible APIs. vLLM supports AMD GPUs and Intel GPUs in addition to NVIDIA, and it is fully open source under the Apache 2.0 license. The 1.x containers wrapped vLLM as one of several backends and added pre-compiled TensorRT-LLM profiles (unavailable in plain vLLM), multi-LoRA dynamic serving, enterprise support contracts, and a curated model catalog vetted by NVIDIA.^[13][14] In the 2.0 generation the relationship is more direct: NVIDIA's documentation calls NIM "an enterprise orchestration layer for vLLM" that packages vLLM with curated model profiles, validated configurations, health management, observability and security hardening, and the differentiation over running vLLM yourself is the tuned configuration bundle, the curated weights and quantized variants, the lifecycle guarantees of NIM Certified, and enterprise support.^[3][30] For teams already comfortable with vLLM who do not need those validated profiles or enterprise SLAs, NIM adds overhead without clear benefit. For teams that want a supported, pre-optimized stack, NIM reduces the configuration burden.

NIM vs. Hugging Face Inference Endpoints: Hugging Face Inference Endpoints provides a managed service for deploying models from the Hugging Face Hub. The service supports multiple GPU providers and does not require an NVIDIA-specific license. It runs TGI or vLLM under the hood and charges per hour of GPU time. NIM offered more GPU-specific optimization (pre-compiled TensorRT-LLM profiles in the 1.x containers, NVIDIA-validated vLLM profiles in 2.0) and a richer model catalog outside the Hub (domain models, speech models, molecular biology models), but requires an NVIDIA AI Enterprise contract for production self-hosting. In June 2024, Hugging Face and NVIDIA announced a collaboration to offer NVIDIA-optimized NIM endpoints through Hugging Face's Serverless Inference API, blending the Hub's model discovery with NVIDIA's inference stack.^[12]

NIM vs. Modal: Modal (platform) offers serverless GPU compute billed per second, with containers that scale to zero when idle. It does not provide a curated model catalog; operators bring their own code and containers. NIM requires persistent infrastructure (or cloud instances) that are always running, which adds cost for sporadic workloads but reduces latency for steady-state production traffic. Modal's flexibility suits research and variable workloads; NIM targets steady-state enterprise production traffic.

What is NIM used for?

Enterprise deployments of NIM cluster around several patterns.

Customer-facing chat and copilots: Organizations integrate NIM-served LLMs into internal helpdesks, customer service interfaces, and productivity copilots. ServiceNow, SAP, and Box were among the enterprise software vendors announced at GTC 2024 as early NIM adopters, integrating NIM-powered inference into their platforms.^[1]

Retrieval-augmented generation pipelines: NIM provides both embedding models (for indexing and query encoding) and LLMs (for answer generation) within the same container ecosystem, making it straightforward to build RAG pipelines where both components are served through the same API format. The NeMo Retriever NIM microservices are specifically tuned for retrieval tasks.

Code generation and software development: Code-specific models like Code Llama 70B are available as NIMs. Enterprises deploy these behind internal developer tools or IDEs, serving code completions and explanations within their security perimeter rather than routing code to third-party APIs.

Healthcare and life sciences: Pharmaceutical companies use BioNeMo NIMs for in silico drug screening, where molecular generation models propose candidate compounds and structure prediction models (ESMFold, DiffDock) filter them. More than 200 organizations were integrating BioNeMo NIMs into their workflows within a year of the platform's launch.

Edge and workstation inference: NIM supports deployment on workstations with RTX 4080 and RTX 4090 GPUs, enabling inference at the edge for applications where sending data to a cloud endpoint is not acceptable due to latency, cost, or data privacy constraints. NVIDIA RTX AI PCs are an explicit supported platform.

Financial services and security: CrowdStrike was among the GTC 2024 launch partners, integrating NIM into its Falcon platform for security analytics.^[1] Financial institutions use NIM-hosted LLMs for document analysis, compliance review, and fraud detection, running inference within their own data centers to satisfy regulatory requirements around data residency.

Manufacturing and digital twins: Siemens, one of the named NIM adopters, explored integrating NIM with its industrial AI toolchain for predictive maintenance and process optimization. NVIDIA's Omniverse platform, which is used for industrial simulation and digital twins, integrates with NIM for language and reasoning tasks within simulation workflows.

Document processing at scale: The multimodal PDF extraction Blueprint lets organizations process large archives of PDFs, extracting text, tables, charts, and figures into structured formats suitable for downstream analysis or RAG pipelines. This use case targets legal, financial, and compliance document workflows where structured extraction from unstructured documents is a persistent bottleneck.

Adoption and ecosystem

At the GTC 2024 announcement, NVIDIA named Adobe, Cadence, CrowdStrike, Getty Images, SAP, ServiceNow, and Shutterstock as early NIM adopters.^[1] By August 2024, when NIM Agent Blueprints launched, the named enterprise customer list had grown to include Lowe's, Siemens, Cohesity, Dropbox, NetApp, and Glean.^[4]

Cloud providers integrated NIM into their AI model catalogs: AWS SageMaker, Google Cloud Vertex AI, and Microsoft Azure AI all offered NIM containers through their platforms by mid-2024.^[11] Oracle Cloud added NIM to Oracle Cloud Infrastructure in a more limited capacity.

Hardware partners including Dell, HPE, Lenovo, and Supermicro validated NIM on their NVIDIA-Certified server lines and began offering NIM as a pre-installed option on AI factory deployments. Cisco validated NIM on its UCS server platforms.

Red Hat OpenShift published guidance for running NIM on OpenShift AI in May 2025, reflecting growing interest in running NIMs within Kubernetes environments that use Red Hat's enterprise Linux distribution.^[15]

NIM 1.4, released in December 2024, achieved 2.4x faster inference than the 1.0 release across several benchmark configurations. DeepSeek-R1 was added as a preview NIM in January 2025 following the model's public release.

Cloudera reported 36x performance improvements after integrating NIM into its data platform, compared to previous inference configurations. This figure reflects both the TensorRT-LLM optimization and the removal of per-request engine compilation overhead that had characterized their earlier setup.

By the end of 2024, NVIDIA said hundreds of enterprises had deployed NIM in production, and the catalog had grown from the initial 20+ models at GTC 2024 to over 100 models.^[1][3] Those remain the figures NVIDIA put on the record for that period; the sections below track what shipped afterwards. The build.nvidia.com platform accumulated millions of API calls from developers using the free hosted endpoints for evaluation.

The NIM ecosystem also expanded through NVIDIA's AI Foundry service, a managed offering that combines NIM with NeMo fine-tuning and model management. AI Foundry targets organizations that want to customize a foundation model with proprietary data and deploy it as a NIM, without managing the fine-tuning infrastructure themselves.

2025-2026 developments

Through 2025 and into 2026, NVIDIA positioned NIM as the deployment layer for a new wave of reasoning and agentic models while expanding the catalog beyond language into media, 3D, and broadcast workflows.

DeepSeek-R1 general availability

The DeepSeek-R1 reasoning model, which had been added as a preview in January 2025, reached general availability as a NIM microservice on build.nvidia.com on January 30, 2025.^[24] NVIDIA reported that the 671-billion-parameter model could deliver up to 3,872 tokens per second on a single NVIDIA HGX H200 system when served through the NIM microservice, and stated that "the DeepSeek-R1 NIM microservice simplifies deployments with support for industry-standard APIs."^[24]

Llama Nemotron reasoning models

At GTC 2025 on March 18, 2025, NVIDIA introduced the Llama Nemotron family of open reasoning models, post-trained from Meta's Llama models to improve multistep math, coding, and complex decision-making for agentic applications.^[18] The models are offered as NIM microservices in three sizes: Nano (tuned for accuracy on PCs and edge devices), Super (highest throughput on a single GPU), and Ultra (maximum agentic accuracy on multi-GPU servers).^[18] NVIDIA stated that the post-training process raises accuracy by up to 20 percent over the base model and improves inference speed by up to 5x relative to other leading open reasoning models.^[18] Early enterprise users building on the Llama Nemotron NIM microservices included Microsoft (Azure AI Foundry and the Azure AI Agent Service), SAP (the Joule copilot and ABAP code completion), ServiceNow, Accenture (its AI Refinery platform), and Deloitte (its Zora AI agentic platform).^[18]

Blackwell-optimized open models

Several major open models released in 2025 were packaged as NIM microservices with optimization for the Blackwell architecture. Meta's Llama 4 Scout and Llama 4 Maverick, announced on April 5, 2025, were made available as NIM microservices; NVIDIA reported that on Blackwell B200 GPUs the optimized Llama 4 Scout exceeds 40,000 tokens per second and Llama 4 Maverick exceeds 30,000 tokens per second, with the B200 delivering 3.4x higher throughput and 2.6x better cost per token than the H200.^[19] On August 5, 2025, OpenAI's open-weight gpt-oss-120b and gpt-oss-20b models launched as NIM microservices, with NVIDIA reporting up to 1.5 million tokens per second for gpt-oss-120b on a single Blackwell GB200 NVL72 rack-scale system.^[20]

Nemotron 3

On December 15, 2025, NVIDIA announced the Nemotron 3 family of open models, built on a hybrid latent mixture-of-experts (MoE) architecture and offered for secure on-premises deployment as NIM microservices.^[25] The family spans three sizes: Nemotron 3 Nano (around 30 billion parameters, activating up to about 3 billion per token), Nemotron 3 Super (around 100 billion parameters), and Nemotron 3 Ultra (around 500 billion parameters).^[25] NVIDIA stated that Nemotron 3 Nano delivers about 4x higher token throughput than Nemotron 2 Nano, reduces reasoning-token generation by up to 60 percent to lower inference cost, and supports a context window of up to 1 million tokens.^[25] Nemotron 3 Nano was made available at announcement, with the Super and Ultra variants expected in the first half of 2026, distributed through Hugging Face and a range of inference providers in addition to NVIDIA NIM microservices.^[25] Both shipped on that schedule: NVIDIA released Nemotron 3 Super on March 11, 2026 as a 120B-total, 12B-active hybrid Mamba-Transformer MoE, and Nemotron 3 Ultra on June 4, 2026 as a 550B-parameter model with 55B active parameters that NVIDIA describes as "built for frontier reasoning and orchestration in agentic systems", with NVFP4 precision, LatentMoE expert routing and multi-token prediction among its architectural features.^[34][35] The Ultra NIM is published on NGC as nvcr.io/nim/nvidia/nemotron-3-ultra-550b-a55b and hosted on build.nvidia.com as nvidia/nemotron-3-ultra-550b-a55b.^[28]

Nemotron 3 Ultra NIM 2.0.12 full-stack optimization case study

On September 10, 2026, NVIDIA published a technical blog post by Arun Venkatesan, Chintan Patel, Adam Shaver and Sungsoo Ha describing how the NIM 2.0.12 serving stack for Nemotron 3 Ultra was tuned, and what that tuning bought on a four-GPU NVIDIA B200 system.^[28] NVIDIA's @NVIDIAAI account summarized the result on September 14 as engineers having "tuned caching, memory, parallelism, decoding and more" so that the optimizations "supported up to 2.5x more concurrent users while maintaining 50 TPS/user."^[36]

Benchmark definition. The post fixes one workload throughout: hardware of 4xB200, and an agentic traffic profile it writes as "64K/400/76% KV reuse/50 TPS/user (20 ms ITL)".^[28] The post does not spell that shorthand out; read in the usual input-length / output-length / cache-reuse / per-user-speed order, it describes 64K-token prompts, 400-token responses, 76 percent KV cache reuse across requests, and an interactivity target of 50 tokens per second per user, which corresponds to a 20 millisecond inter-token latency. NVIDIA argues that this tradeoff between concurrency and interactivity matters most for agentic AI workloads, where prompts are long, context is reused across steps, and responses are streamed.^[28]

Result. The post compares what it calls "the open-source baseline serving stack (NIM Off)" against "the fully optimized NIM 2.0.12 serving stack (NIM On)" on a Pareto chart of system throughput against per-user speed, and reports the values at the 50 TPS/user target in a table whose throughput column is labeled "Native 256K max context":^[28]

NVIDIA presents the gain as "up to 2.5x" throughout and, in the figure caption, as "more than 2.5x the system throughput of the baseline, translating directly into more concurrent users at the same interactivity."^[28] The two table figures divide to about 2.8 (a calculation from the published table, not a figure NVIDIA states); the "more users" framing follows because, at a fixed 50 TPS per user, higher aggregate throughput means more simultaneous users can be served at that speed.^[28]

Optimization layers. The post is explicit that "the measured gains come from interacting configuration bundles, not independent switches whose percentages can simply be added," and it does not attribute a percentage to any single layer.^[28] The layers it lists are:

The deployment recipe that goes with these numbers pins the image tag to 2.0.12, selects the vllm-nvidia-b200-nvfp4-tp4-pp1-throughput-90.0 profile through NIM_MODEL_PROFILE, and sets NIM_SPECDEC_ENABLE=1.^[28] The profile string therefore records the backend and precision behind the headline figure: a vLLM backend serving NVFP4 weights with four-way tensor parallelism.^[28]

Benchmarking with NVIDIA AIPerf. NVIDIA frames the published curves as "a starting point, not a promise that every application will see the same result," and recommends a four-step method: deploy the exact software version (NIM 2.0.12 or newer, with the image tag or digest pinned for every run); prepare representative traffic, either a Mooncake-format JSONL trace or captured NIM requests, with access controls and sanitization for sensitive data; replay that traffic with NVIDIA AIPerf; and then compare output throughput only among the measured points that satisfy the latency SLO, choosing the Pareto point that fits.^[28] The example sweep in the post loops aiperf profile over concurrency levels of 1, 4, 8, 16, 32 and 64 against the chat endpoint in streaming mode, feeding the trace through --custom-dataset-type mooncake_trace with --no-fixed-schedule.^[28] AIPerf itself is an Apache-2.0 benchmarking tool published in the ai-dynamo GitHub organization (the same organization as NVIDIA Dynamo); its README describes concurrency, request-rate and trace-replay benchmarking modes, a multiprocess architecture, and a plugin system for endpoints, datasets and metrics.^[33]

Caveats. The comparison is a vendor benchmark of NVIDIA's own software against a baseline NVIDIA defines. The post names the baseline only as "the open-source baseline serving stack" and does not identify which engine or version it is, whether it was the same vLLM release the NIM builds on, or how it was configured; the figures are NVIDIA's own measurements on NVIDIA hardware, for one model and one synthetic traffic profile.^[28] NVIDIA's closing section promises "more performance-optimized NIM configurations" across a broader range of models, which is a statement of intent rather than a shipped feature.^[28]

GTC 2026 and the agentic stack

At GTC 2026, held in San Jose from March 16 to March 19, 2026, NVIDIA continued to frame NIM as a core part of its agentic AI stack, with NIM microservices and Nemotron open models cited as the software foundation for autonomous agent and digital-human deployments shown during the event.^[26]

Media and broadcast NIM microservices

In a 2026 update to its AI for Media (Maxine) product line, NVIDIA released a set of SMPTE ST 2110-compliant NIM microservices, namely LipSync, Active Speaker Detection, Studio Voice, and Video Super Resolution, designed to run inside live IP-based broadcast pipelines, together with a Synthetic Video Detector NIM for flagging AI-generated footage in newsroom and content-platform workflows.^[22][23]

What are NIM's limitations?

NIM has several meaningful constraints for organizations evaluating it.

NVIDIA hardware exclusivity. NIM containers only run on NVIDIA GPUs. Organizations with AMD or Intel GPU investments, or those looking to hedge against hardware vendor concentration, cannot use NIM on non-NVIDIA hardware. vLLM and Triton Inference Server both support AMD ROCm GPUs.

Production licensing cost. The $4,500 per GPU per year subscription cost is substantial for small organizations or projects with limited GPU counts.^[6] For a small four-GPU inference server, the annual license fee alone is $18,000, before accounting for hardware and cloud costs. Open-source alternatives like vLLM have no licensing cost.

Limited to NVIDIA's model catalog. While the catalog covers most popular open-weight models, it does not include every model available on Hugging Face. Less popular or recently released models may not yet have a NIM, requiring teams to fall back to other serving frameworks for those models.

NVIDIA-curated optimization profiles. In the 1.x containers the pre-compiled TensorRT-LLM profiles were compiled artifacts distributed by NVIDIA rather than open source, and users could not inspect or modify the compilation process. The 2.0 containers are built on open-source vLLM, but the validated profiles, curated quantized weights and tuned configuration bundles that carry the performance claims are still produced and chosen by NVIDIA, and only for the model, GPU and precision combinations it decides to validate.^[3][28][31]

Vendor-defined benchmarks. NVIDIA's performance claims for NIM, from the 2024 Llama 3.1 8B comparison to the 2026 Nemotron 3 Ultra "up to 2.5x more users" result, are NVIDIA's own measurements of NVIDIA software on NVIDIA hardware against baselines that NVIDIA defines. In the Nemotron 3 Ultra case the baseline is described only as "the open-source baseline serving stack", without an engine name, version or configuration, and the workload is a single synthetic agentic trace.^[2][28] NVIDIA itself says the curves are "a starting point, not a promise," and the honest way to evaluate NIM against an alternative is the method NVIDIA recommends: replay your own traffic with a tool such as AIPerf against both stacks and compare only the points that meet your latency target.^[28]

Dependency on NVIDIA infrastructure. Pulling most container images requires authenticating to NVIDIA's NGC registry, although the 2.0 documentation says eligible public-catalog NIMs can be pulled and run without an NGC API key.^[3] If NGC has downtime, or if NVIDIA changes access terms, pulling updated images or deploying to new infrastructure becomes harder. Organizations with strict supply-chain requirements may find this dependency difficult to accept.

See also

• Snowflake-AWS chip deal
• TensorRT
• NVIDIA TensorRT-LLM
• NVIDIA Triton Inference Server
• NVIDIA Dynamo
• NVIDIA AI Enterprise
• NVIDIA NeMo
• vLLM
• Hugging Face
• Modal (platform)
• LangChain
• LlamaIndex
• NVIDIA DGX
• CUDA
• Retrieval-augmented generation

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8NVIDIA Newsroom. "NVIDIA Launches Generative AI Microservices for Developers." March 18, 2024. nvidianews.nvidia.com/...roservices-for-developers
2. ^1 ^2NVIDIA Technical Blog. "NVIDIA NIM Offers Optimized Inference Microservices for Deploying AI Models at Scale." 2024. developer.nvidia.com/...ploying-ai-models-at-scale
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15NVIDIA Documentation. "Overview, NVIDIA NIM for Large Language Models (LLM) and Vision Language Models (VLM)." docs.nvidia.com/...overview
4. ^1 ^2 ^3 ^4NVIDIA Newsroom. "NVIDIA and Global Partners Launch NIM Agent Blueprints for Enterprises to Make Their Own AI." August 27, 2024. nvidianews.nvidia.com/...ises-to-make-their-own-ai
5. ^1 ^2NVIDIA Blog. "NIM Agent Blueprints Fast-Forward Next Wave of Enterprise Generative AI." 2024. blogs.nvidia.com/...nim-agent-blueprints
6. ^1 ^2 ^3 ^4 ^5 ^6 ^7NVIDIA Documentation. "NVIDIA AI Enterprise Pricing." docs.nvidia.com/...pricing
7. ^NVIDIA Documentation. "Fine-Tuning with LoRA, NVIDIA NIM for LLM and VLM." docs.nvidia.com/...finetune-lora
8. ^NVIDIA Technical Blog. "Seamlessly Deploying a Swarm of LoRA Adapters with NVIDIA NIM." developer.nvidia.com/...a-adapters-with-nvidia-nim
9. TechCrunch. "Nvidia launches NIM to make it smoother to deploy AI models into production." March 18, 2024. techcrunch.com/...rvices-for-optimized-inferencing
10. ^1 ^2NVIDIA Newsroom. "NVIDIA Healthcare Launches Generative AI Microservices to Advance Drug Discovery, MedTech and Digital Health." 2024. nvidianews.nvidia.com/...nerative-ai-microservices
11. ^1 ^2AWS Blog. "Get Started with NVIDIA NIM Inference Microservices on Amazon SageMaker." aws.amazon.com/...icroservices-on-amazon-sagemaker
12. ^Hugging Face Blog. "Serverless Inference with Hugging Face and NVIDIA NIM." huggingface.co/...inference-dgx-cloud
13. ^NVIDIA Developer Forums. "vLLM vs NVIDIA NIM." forums.developer.nvidia.com/...357164
14. ^1 ^2Dell Technologies. "Tailoring LLM Inference with NVIDIA NIM using Key Features of TensorRT-LLM and vLLM." infohub.delltechnologies.com/...sorrt-llm-and-vllm
15. ^Red Hat Developer. "How to set up NVIDIA NIM on Red Hat OpenShift AI." May 2025. developers.redhat.com/...-nim-red-hat-openshift-ai
16. ^1 ^2 ^3 ^4NVIDIA Newsroom. "NVIDIA Dynamo Open-Source Library Accelerates and Scales AI Reasoning Models." March 18, 2025. nvidianews.nvidia.com/...cales-ai-reasoning-models
17. ^1 ^2 ^3 ^4NVIDIA Technical Blog. "Introducing NVIDIA Dynamo, A Low-Latency Distributed Inference Framework for Scaling Reasoning AI Models." March 2025. developer.nvidia.com/...caling-reasoning-ai-models
18. ^1 ^2 ^3 ^4 ^5 ^6NVIDIA Newsroom. "NVIDIA Launches Family of Open Reasoning AI Models for Developers and Enterprises to Build Agentic AI Platforms." March 18, 2025. nvidianews.nvidia.com/...uild-agentic-ai-platforms
19. ^1 ^2NVIDIA Technical Blog. "NVIDIA Accelerates Inference on Meta Llama 4 Scout and Maverick." April 5, 2025. developer.nvidia.com/...llama-4-scout-and-maverick
20. ^1 ^2NVIDIA Blog. "OpenAI and NVIDIA Propel AI Innovation With New Open Models Optimized for the World's Largest AI Inference Infrastructure." August 5, 2025. blogs.nvidia.com/...openai-gpt-oss
21. ^1 ^2 ^3NVIDIA Newsroom. "NVIDIA Announces Generative AI Models and NIM Microservices for OpenUSD Language, Geometry, Physics and Materials." July 29, 2024. nvidianews.nvidia.com/...microservices-for-openusd
22. ^1 ^2 ^3NVIDIA Developer Forums. "AI for Media Product Update: New Real-Time AI Models and Infrastructure Microservices." June 2, 2026. forums.developer.nvidia.com/...372024
23. ^1 ^2 ^3NVIDIA Documentation. "Overview, NVIDIA NIM Maxine Synthetic Video Detector." docs.nvidia.com/...overview
24. ^1 ^2NVIDIA Blog. "DeepSeek-R1 Now Live With NVIDIA NIM." January 30, 2025. blogs.nvidia.com/...deepseek-r1-nim-microservice
25. ^1 ^2 ^3 ^4NVIDIA Newsroom. "NVIDIA Debuts Nemotron 3 Family of Open Models." December 15, 2025. nvidianews.nvidia.com/...n-3-family-of-open-models
26. ^NVIDIA Blog. "NVIDIA GTC 2026: Live Updates on What's Next in AI." March 2026. blogs.nvidia.com/...gtc-2026-news
27. ^NVIDIA Technical Blog. "Chat With Your Enterprise Data Through Open-Source AI-Q NVIDIA Blueprint." June 11, 2025. developer.nvidia.com/...urce-ai-q-nvidia-blueprint
28. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22NVIDIA Technical Blog. "How Full-Stack NIM Optimizations Deliver 2.5x More Users on Nemotron 3 Ultra." September 10, 2026. developer.nvidia.com/...-users-on-nemotron-3-ultra
29. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10NVIDIA Documentation. "NIM LLM 1.x to NIM LLM and VLM 2.0 Migration Guide." docs.nvidia.com/...1.x-migration-guide
30. ^1 ^2 ^3 ^4 ^5 ^6 ^7NVIDIA Documentation. "Architecture, NVIDIA NIM for Large Language Models (LLM) and Vision Language Models (VLM)." docs.nvidia.com/...architecture
31. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9NVIDIA Documentation. "Model Profiles and Selection, NVIDIA NIM for LLM and VLM." docs.nvidia.com/...model-profiles-and-selection
32. ^1 ^2 ^3 ^4NVIDIA Documentation. "NIM Offerings, NVIDIA NIM for LLM and VLM." docs.nvidia.com/...nim-offerings
33. ^ai-dynamo. "AIPerf." GitHub repository (Apache-2.0). github.com/...aiperf
34. ^NVIDIA Technical Blog. "Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning." March 11, 2026. developer.nvidia.com/...-moe-for-agentic-reasoning
35. ^NVIDIA Technical Blog. "NVIDIA Nemotron 3 Ultra Powers Faster, More Efficient Reasoning for Long-Running Agents." June 4, 2026. developer.nvidia.com/...ng-for-long-running-agents
36. ^NVIDIA AI (@NVIDIAAI) on X. "A lot of work goes into serving a model efficiently..." September 14, 2026. x.com/...2099550869381353910
37. ^NVIDIA Documentation. "Release Notes, NVIDIA NIM for Large Language Models 3.0.0." docs.nvidia.com/...release-notes
38. ^NVIDIA. "NVIDIA AI Enterprise." nvidia.com/...ai-enterprise