NVIDIA Exemplar Cloud
NVIDIA Exemplar Cloud is a validation program run by NVIDIA that certifies cloud providers whose GPU clusters reproduce at least 95% of the training throughput NVIDIA measures on its own reference architecture for the same set of large-model workloads. NVIDIA announced the initiative, originally styled "Exemplar Clouds", at Computex in Taipei in May 2025 [1][2]. On 30 July 2026 NVIDIA's developer blog published an unusually specific engineering post that walked through four real partner deployments which had missed the threshold, naming the root cause and the exact configuration change that closed each gap [3].
The post is notable because it quantifies something vendors rarely put in writing: two clusters built from identical NVIDIA H100, GB200 NVL72, or GB300 NVL72 systems routinely differ by 8% to 12% in training throughput on the same workload, the same model, and the same global batch size, and the difference comes from configuration rather than silicon [3].
| Field | Value |
|---|---|
| Program name | NVIDIA Exemplar Cloud (announced as "NVIDIA Exemplar Clouds") |
| Announced | 19 May 2025, Computex, Taipei [1][2] |
| Operator | NVIDIA |
| Pass threshold | Greater than 95% of NVIDIA baseline performance on every workload in the chip's test suite [4] |
| Test scale | 512 GPUs per workload, one run each, profiling disabled [4] |
| Tooling | NVIDIA Performance Benchmarking Recipes (NVIDIA/dgxc-benchmarking), run under Slurm via llmb-run exemplar [4][5] |
| Chip suites | GB300, GB200, B300, B200, H100 [4] |
| Validity | 12 months, with periodic re-runs against updated baselines [4] |
| Listed providers | 14, as of 31 July 2026 [6] |
Why identical hardware does not mean identical performance
An AI training cluster is a stack of dependent layers, and each one has defaults that were chosen for some other kind of workload. Server firmware ships with power profiles tuned for latency-sensitive web serving. Hypervisors interpose on device memory mapping. Container runtimes drop environment variables that the host had set. The interconnect library picks routing parameters from an autodetected topology that may be wrong inside a virtual machine.
None of these individually looks like a failure. A GPU cluster passes acceptance tests, nvidia-smi reports healthy clocks, and single-node throughput matches spec. The loss appears only at scale and only under a real workload, which is why NVIDIA's engineers describe the gaps as cumulative: a few percent from CPU power settings, a few more from process placement, more again from a missing kernel capability or an unpropagated topology file, compounding into a number large enough to fail validation [3].
The commercial stakes are direct. GPU capacity is sold by the hour, and a cluster running 10% slow costs its tenant roughly 10% more for the same finished model. That has made measurable, reproducible performance a competitive axis among neoclouds and hyperscalers alike, alongside third-party rating systems such as SemiAnalysis's ClusterMAX, which scores GPU clouds on ten operational criteria including networking, reliability, and pricing [7].
The program
Origin
NVIDIA introduced Exemplar Clouds alongside DGX Cloud Lepton at Computex on 19 May 2025, framing it as a way to help NVIDIA Cloud Partners (NCPs) improve security, usability, performance, and resiliency using NVIDIA's reference hardware and software and operational tools [1]. NCPs are cloud providers that build AI infrastructure to NVIDIA-validated designs and sit inside the NVIDIA Partner Network [25]. A same-week developer blog by Abhishek Sinha, senior director of product at NVIDIA, described the problem it was meant to solve: there had been "no standardized or public way to verify that these platforms are tuned for real-world AI workloads or exhibit resiliency needed for developer productivity" [2]. Yotta Data Services of India was named the first Asia-Pacific NCP to join [1][2].
The program sits on top of two existing pieces of NVIDIA infrastructure. The first is the NCP reference architecture, a blueprint published in June 2024 covering GPU servers, InfiniBand or Spectrum-X Ethernet networking, certified high-performance storage, and the NVIDIA AI Enterprise software stack [8]. The second is DGX Cloud Benchmarking, a suite of containerized "recipes" released in March 2025 that measure training and inference performance end to end and report cost and time to train, not just raw FLOPS [9]. Exemplar validation is essentially the NCP reference architecture used as a scoring baseline, with the benchmarking recipes as the instrument.
What validation involves
The requirements are published openly in the NVIDIA/dgxc-benchmarking repository. A provider must show greater than 95% of baseline performance on every workload in the test suite for the chip generation being validated [4]. Suites run at 512 GPUs, one repeat per workload, profiling disabled, on a Slurm cluster using Enroot containers and the Pyxis plugin [4][5]. The process runs in five stages: kickoff with the NVIDIA account team, cluster preparation and a prescreen test, execution via llmb-run exemplar, submission of a .tar.zst results archive, and a joint optimization pass before qualification. Certification lasts 12 months [4].
Recipes are re-released roughly every two months as model architectures change, and a provider must stay on the same recipe version for the duration of its certification [4].
| Chip suite | Models in the suite (parameter count) | Precisions |
|---|---|---|
| GB300 | DeepSeek-V3 671B, GPT-OSS 120B, Kimi-K2 1T, Llama 3.1 405B and 70B, Nemotron-H 56B, Nemotron 3 120B, Qwen3 235B | BF16, FP8, NVFP4 |
| GB200 | Same eight models | BF16, FP8, NVFP4 |
| B300 | Same eight models | BF16, FP8, NVFP4 |
| B200 | Same eight models | BF16, FP8, NVFP4 |
| H100 | GPT-OSS 120B, Llama 3.1 70B, Nemotron-H 56B, Qwen3 235B | BF16, FP8 |
Source: NVIDIA Exemplar validation requirements, retrieved 31 July 2026 [4].
Validated providers
NVIDIA maintains the roster on its AI cloud performance page. As of 31 July 2026 it listed 14 providers, with validation recorded per chip generation rather than per company.
| Provider | Validated generations (as of 31 July 2026) |
|---|---|
| AWS | GB300 |
| Boost Run | B300 |
| CoreWeave | H100, GB200 |
| Crusoe | B200 |
| Global AI | GB300 |
| IREN | B300 |
| Lambda | H100 |
| Microsoft Azure | H100, GB300 |
| Nebius | H200, GB300 |
| Nscale | GB300 |
| Oracle Cloud Infrastructure | B200, B300, GB200, GB300 |
| VisionBay | B300 |
| Vultr | B200 |
| YTL | GB200 |
Source: NVIDIA Exemplar Cloud partner table, retrieved 31 July 2026 [6].
Lambda announced its H100 validation on 9 October 2025 [10], and Nebius reported clearing the H200 suite on 29 September 2025 with "more than 97% of the performance benchmark" [11]. Microsoft said on 18 February 2026 that Azure was the first provider recognized for GB300-class systems, building on an earlier H100 validation [12]. Vultr (7 April 2026) [13], Boost Run (13 April 2026) [14], Nebius on GB300 (29 April 2026) [15], YTL AI Cloud (31 May 2026) [16], Crusoe (1 June 2026) [17], IREN (29 June 2026) [18], and Nscale (16 July 2026) [19] followed. On 31 May 2026 NVIDIA's Dion Harris wrote that six NCPs had achieved the status since launch: CoreWeave, Crusoe, Lambda, Nebius, Vultr and YTL [20], a count that excludes the hyperscalers on the public table.
Most validations to date cover training. CoreWeave said on 30 July 2026 that it had extended its GB200 NVL72 validation to inference, testing chat, reasoning, summarization, generation, and disaggregated serving phases and reporting tokens per second per GPU and time to first token [21].
The July 2026 engineering post
The 30 July 2026 post, by Emily Potyraj, Pavan Sridhar, Sriharsha Niverty, Suryakant Patidar and Charlie Huang, is a debugging walkthrough rather than a marketing piece [3]. Potyraj is NVIDIA's product manager for Exemplar Cloud performance and benchmarking; Sridhar leads the Exemplar performance program for cloud AI infrastructure [3]. Each case isolates a different layer of the stack.
| # | Platform | Layer | Diagnostic signal | Fix | Recovered |
|---|---|---|---|---|---|
| 1 | GB200 NVL72 (VM) | Arm SMMU | arm_smmu_cmdq_issue_cmdlist dominant in perf; multi-fold dTLB miss increase | Enable CMDQV/VCMDQ in host kernel and expose to guest | about 12% |
| 2 | H100 (VM) | CPU power and NUMA | Cores stuck at 3.0 GHz; bimodal step time; 18% NUMA-remote access | C-state tuning, cpuset isolation, numactl binding, SMT and mitigations off | 9% (12 to 3) |
| 3 | GB300 NVL72 | NCCL queue-pair concurrency | AllGather bus bandwidth about 28 GB/s versus about 61 GB/s with QPS=4 | NCCL_IB_QPS_PER_CONNECTION from 1 to 4 on ConnectX-8 | 31% of iteration time |
| 4 | B200 (VM) | Runtime-visible topology | Host topology correct, but NCCL_TOPO_FILE and /etc/nccl/topo.xml absent inside the container; collectives 2 to 4 times slower | Bind-mount the topology file into the job container | Closed the 13% to 53% gap |
Source: NVIDIA, summary of fixes, 30 July 2026 [3]. NVIDIA's own table spells the case 1 fix "VMDQV", which conflicts with the CMDQV/VCMDQ used throughout the article body and appears to be a typographical error.
Case 1: virtualization overhead on Grace
A GB200 NVL72 partner deployment running DeepSeek-V3 Mixture-of-Experts FP8 pre-training inside a virtual machine produced iteration times 12% to 14% longer than the bare-metal reference. Dense recipes such as Llama 3 70B ran within 3% of reference; the MoE workload, which issues many small kernels per iteration, was the outlier [3].
A 30-second perf record -a -g capture on the host put 24% of CPU cycles in arm_smmu_cmdq_issue_cmdlist, the function that submits invalidation commands to the Arm system memory management unit's command queue. Under virtualization every guest invalidation traps to the host and serializes through a single queue. The fix was to enable Command Queue Virtualization, an extension to Arm SMMUv3 that gives a guest its own virtual command queues so it can issue invalidations directly to hardware without VM exits [3]. Support reached the Linux kernel as the tegra241-cmdqv driver, whose author reported up to a 20% reduction in TLB invalidation time for in-kernel use and 70% to 90% reductions for guests compared with a trapped nested SMMU command queue [22]. NVIDIA notes that recent QEMU and libvirt versions expose the capability to guests through a cmdqv IOMMU attribute [3].
Case 2: CPU power and process placement
An H100 SXM5 cluster running the same NCCL version and NeMo container as NVIDIA's HGX reference was 12% slower on Llama 3 70B pre-training. turbostat showed busy cores pegged at 3.0 GHz on a part rated for 3.8 GHz turbo, with idle cores held in C1 rather than dropping to C6 because BIOS limited C-states, a common low-latency default. Idle cores in C1 keep drawing package power, so the busy cores feeding the GPUs could not claim enough of the power budget to reach turbo. Allowing C6 recovered roughly 4% [3].
Separately, numastat showed about 18% of the training process's memory accesses going to the remote NUMA node, and hypervisor housekeeping threads were pinned to the same physical cores as the data loader workers, producing sporadic 50 to 100 ms stalls that showed up as a long tail in step time. Separating them with cpusets (host services on cores 0 to 7 and 56 to 63, training on the rest) plus NUMA binding took the total gap from 12% to 3% [3].
Case 3: queue-pair concurrency on a 1.6 Tbps fabric
A GB300 NVL72 deployment with NVIDIA ConnectX-8 SuperNICs at 1.6 Tbps per node showed a 31% gap on Nemotron-4 15B pre-training. Single-node throughput was healthy; the loss appeared at 512 GPUs, with profiler traces showing exposed AllGather and ReduceScatter time. After testing iteration count, UCX/UCC behavior, NUMA mapping, NVLS, and NCCL versions, the change that mattered was raising NCCL_IB_QPS_PER_CONNECTION from its default of 1 to 4 [3]. NCCL documents that variable as the number of InfiniBand queue pairs used for each connection between two ranks, accepting values from 1 to 128, and notes it can help on multi-level fabrics that need multiple queue pairs for routing entropy [23].
On NVIDIA's reference cluster the default configuration ran at about 1.09 seconds per iteration and QPS=4 improved it to about 0.83 seconds, with AllGather falling from about 375 ms to 262 ms and ReduceScatter from about 389 ms to 273 ms. (These iteration times are NVIDIA's reference-cluster measurements, not the partner's.) A separate comparison run at about 0.76 seconds used a different NCCL version, so part of the residual was a version mismatch rather than tuning; because NCCL version changes fall outside normal Exemplar tuning scope, NVIDIA's recommendation keeps the deployed version fixed [3]. NVIDIA cautions against raising the setting everywhere: on other fabrics or message-size profiles it can add CPU overhead without improving throughput [3].
Case 4: the environment variable that never made it inside
In a virtualized B200 deployment, throughput was 13% to 53% below reference even though nccl-tests on the host looked normal. Inside the Enroot workload container, AllGather and ReduceScatter were 2 to 4 times slower. Neither NCCL_TOPO_FILE nor the /etc/nccl/topo.xml it pointed at had been propagated into the container, so NCCL silently fell back to autodetection with no error emitted. Bind-mounting the file closed the gap [3]. NVIDIA's advice is to run checks from inside the same container, launcher, and Slurm allocation that will run the benchmark, and treats echo $NCCL_TOPO_FILE && cat $NCCL_TOPO_FILE inside the job container as the fastest sanity check [3].
Preflight checks and tools
| Area | What to check | Tools |
|---|---|---|
| GPU and hardware health | Clock, power, thermal, and NVLink bandwidth consistency under sustained load | nvidia-smi, DCGM, dcgm-exporter |
| Grace and VM readiness | CMDQV support, guest page size, IOMMU passthrough, large-page availability | perf, dmesg, kernel config, boot parameters |
| CPU power and placement | Busy-core turbo, cpuset isolation, NUMA and PCT binding near GPUs | turbostat, lscpu, numactl, nvidia-smi topo -m |
| Runtime topology | Topology files, NCCL environment variables, HCA visibility inside the job container | env, cat $NCCL_TOPO_FILE, NCCL_DEBUG=INFO |
| Fabric collectives | AllGather and ReduceScatter at the workload's real message sizes | nccl-tests, workload traces |
| Workload tuning | Pipeline parallelism, microbatch sizing, communication overlap, only after platform issues are ruled out | Nsight Systems, workload logs |
Source: NVIDIA, recommended preflight checks, 30 July 2026 [3].
NVIDIA is explicit that passing these checks does not guarantee validation, because some problems surface only under the exact model, precision, topology, container, launcher, and network conditions of the validation run [3].
Terms in plain language
A reference architecture is NVIDIA's own build of a cluster, down to firmware settings and software versions, whose measured throughput becomes the baseline everyone else is scored against. NCCL is the library that moves gradients between GPUs during training; collectives are its group operations, of which AllGather (every GPU ends up with every other GPU's slice of data) and ReduceScatter (the sum across GPUs is computed and each GPU keeps one slice of the result) dominate large-model training time. A queue pair is the send and receive queue pair that an InfiniBand or RoCE adapter uses for one connection; using several per connection spreads traffic across more network paths. The IOMMU, called the SMMU on Arm, translates device addresses to physical memory and must be told when mappings change, which is the invalidation traffic that dominated case 1. NUMA locality describes whether a process's memory sits on the memory controller attached to its own CPU socket or across a slower interconnect. C-states are CPU idle power states, where deeper states save more power but take longer to wake from. Enroot and Pyxis are NVIDIA's unprivileged container runtime and its Slurm plugin, used to run benchmark containers on HPC schedulers [5][24].
Scope and limits of the validation
Exemplar Cloud measures a provider against a baseline NVIDIA defines, using recipes NVIDIA publishes, and the passing results providers announce are their own. Neither NVIDIA nor the validated providers publish the raw submission archives, so a third party cannot re-derive a provider's numbers from published artifacts the way MLPerf submissions allow. As of 31 July 2026 none of the outlets that routinely cover accelerated-computing infrastructure had published an independent analysis of the July 2026 case-study post.
The scope is also narrower than "cluster quality" in general. Validation runs a fixed workload suite at 512 GPUs with profiling disabled, so it does not directly measure storage throughput, multi-tenant behavior, failure rates over long runs, or job-scheduling overhead, and a certificate is valid for 12 months against recipes that are re-released roughly every two months. For operators of AI data centers, the concrete content of the July 2026 post is the list of misconfigurations it names: C-states pinned in BIOS, hypervisor threads sharing cores with the training process, a topology file present on the host but not inside the container, and an InfiniBand queue-pair default carried over from a slower generation of fabric.
See also
- NVIDIA
- NVIDIA DGX Cloud
- NCCL
- NVIDIA GB300 NVL72
- Neocloud
- Distributed training
- Cloud GPU pricing comparison
- NVIDIA Blackwell
References
- ^"NVIDIA Announces DGX Cloud Lepton to Connect Developers to NVIDIA's Global Compute Ecosystem." NVIDIA press release, 19 May 2025. investor.nvidia.com/...default
- ^Abhishek Sinha. "Announcing NVIDIA Exemplar Clouds for Benchmarking AI Cloud Infrastructure." NVIDIA Technical Blog, 18 May 2025. developer.nvidia.com/...ng-ai-cloud-infrastructure
- ^Emily Potyraj, Pavan Sridhar, Sriharsha Niverty, Suryakant Patidar and Charlie Huang. "NVIDIA Exemplar Cloud: Lessons for Unlocking Full Performance on AI Infrastructure." NVIDIA Technical Blog, 30 July 2026. developer.nvidia.com/...mance-on-ai-infrastructure
- ^"NVIDIA Exemplar Cloud: Path and Requirements." NVIDIA/dgxc-benchmarking, GitHub, retrieved 31 July 2026. github.com/...Exemplar_validation.md
- ^"DGX Cloud Benchmarking: Performance Recipes." NVIDIA/dgxc-benchmarking README, GitHub, retrieved 31 July 2026. github.com/...dgxc-benchmarking
- ^"NVIDIA Exemplar Cloud." NVIDIA, retrieved 31 July 2026. nvidia.com/...ai-cloud-performance
- ^"ClusterMAX Overview: GPU Cloud Rating Methodology." SemiAnalysis, ratings version 2.1, April 2026. clustermax.ai/overview
- ^Marc Hamilton. "NVIDIA Unveils Reference Architecture for AI Cloud Providers." NVIDIA Blog, 26 June 2024. blogs.nvidia.com/...oviders-reference-architecture
- ^Emily Potyraj. "Measure and Improve AI Workload Performance with NVIDIA DGX Cloud Benchmarking." NVIDIA Technical Blog, 18 March 2025. developer.nvidia.com/...dia-dgx-cloud-benchmarking
- ^"Lambda Achieves NVIDIA Exemplar Cloud." Lambda, 9 October 2025. lambda.ai/...lambda-achieves-nvidia-exemplar-cloud
- ^"Nebius achieves NVIDIA Exemplar Status on NVIDIA H200 GPUs for training workload." Nebius, 29 September 2025. nebius.com/...-nvidia-exemplar-status-on-h200-gpus
- ^Fernando Aznar. "Azure Recognized as an NVIDIA Cloud Exemplar, Setting the Bar for AI Performance in the Cloud." Microsoft Azure High Performance Computing Blog, 18 February 2026. techcommunity.microsoft.com/...4495747
- ^"Vultr Achieves NVIDIA Exemplar Cloud for Surpassing AI Training Performance Standards." Vultr, 7 April 2026. blogs.vultr.com/nvidia-exemplar-cloud-ai-training
- ^"Boost Run Achieves NVIDIA Exemplar Cloud on NVIDIA Blackwell Architecture." PR Newswire, 13 April 2026. prnewswire.com/...infrastructure-leaders-302740029
- ^"Nebius achieves NVIDIA Exemplar Cloud on NVIDIA GB300 for training." Nebius, 29 April 2026. nebius.com/...r-cloud-on-nvidia-gb300-for-training
- ^"YTL AI Cloud Among Asia's First Cloud Providers to Achieve NVIDIA Exemplar Cloud Status." YTL AI Cloud, 31 May 2026. ytlaicloud.com/...eve-nvidia-exemplar-cloud-status
- ^"Crusoe Recognized as an NVIDIA Exemplar Cloud." Crusoe, 1 June 2026. crusoe.ai/...ecognized-as-an-nvidia-exemplar-cloud
- ^"IREN Achieves NVIDIA Exemplar Cloud on NVIDIA HGX B300 for Training." IREN, 29 June 2026. iren.com/...-exemplar-cloud-nvidia-hgx-b300-status
- ^"Nscale achieves NVIDIA Exemplar Cloud status on NVIDIA GB300 NVL72." Nscale, 16 July 2026. nscale.com/...r-cloud-status-on-nvidia-gb300-nvl72
- ^Dion Harris. "NVIDIA AI Cloud Ecosystem Expands Worldwide to Meet Global AI Compute Demand." NVIDIA Blog, 31 May 2026. blogs.nvidia.com/...ai-cloud-ecosystem
- ^"CoreWeave Becomes One of the First Cloud Providers to Achieve NVIDIA Exemplar Cloud Validation for Inference on NVIDIA GB200 NVL72." CoreWeave, 30 July 2026. coreweave.com/...r-inference-on-nvidia-gb200-nvl72
- ^Nicolin Chen. "Add Tegra241 (Grace) CMDQV Support (part 1/2)." LWN.net, 12 June 2024. lwn.net/...978294
- ^"NCCL Environment Variables." NVIDIA Collective Communications Library documentation, retrieved 31 July 2026. docs.nvidia.com/...env
- ^"NVIDIA/enroot." GitHub, retrieved 31 July 2026. github.com/...enroot
- ^"NVIDIA Cloud Partners Deliver Scalable AI Solutions." NVIDIA, retrieved 31 July 2026. nvidia.com/...partners
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.
v1 · 3,336 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify