Citation and evidence

Multi-Instance GPU

16 min full readUpdated 22 references

This article's verification

Report a problem with this article

Edit article
More

Use this article

Raw MarkdownExplore connections

Improve this page

Revision historyDiscuss this article

Topics

AI HardwareAI InferenceAI InfrastructureNVIDIA

Cite this article

Multi-Instance GPU (MIG) is a hardware partitioning feature of NVIDIA GPUs, starting with the Ampere-generation A100, that splits one physical GPU into as many as seven separate GPU instances, each with its own compute resources and its own share of the memory system.[1][8] NVIDIA describes the partitions as isolated: the on-chip crossbar ports, L2 cache banks, memory controllers and DRAM address buses are each assigned to a single instance, so a workload keeps the same cache allocation and memory bandwidth even when a neighbouring instance saturates its own.[1] The feature was aimed first at cloud providers and other multi-tenant operators who want to run several smaller jobs, such as low-batch inference, on one large accelerator without the jobs interfering with each other.[1][8]

NVIDIA introduced MIG with the A100 in May 2020, gave the H100 a second-generation version in 2022, and as of November 2026 supports it on Ampere, Hopper and Blackwell data center GPUs, on RTX PRO Blackwell professional cards and on the integrated GPU of Jetson Thor.[2][8][9] MIG is managed through the NVIDIA driver, the nvidia-smi tool and the NVIDIA Management Library, and it works with containers, Kubernetes and virtual machines.[1][7]

History

NVIDIA announced MIG on May 14, 2020 as a new feature of the A100 Tensor Core GPU, which is based on its Ampere architecture.[8] The launch description said an A100 could be "partitioned into as many as seven separate GPU Instances for CUDA applications," and pitched the feature mainly at cloud service providers, claiming they could offer "up to 7x more GPU Instances for no additional cost."[8] At launch, MIG worked under Linux, with containers through runtimes such as Docker Engine, and with hypervisors; NVIDIA said Kubernetes support was "coming soon."[8] The Ampere-based A30 also supports MIG, with up to four instances.[2]

The Hopper architecture, announced on March 22, 2022, brought what NVIDIA called second-generation MIG. According to NVIDIA, each H100 GPU instance has about three times the compute capacity and nearly twice the memory bandwidth of an A100 instance, each of the up to seven instances has dedicated NVDEC video decoders and NVJPG units, each instance has its own performance monitors, and MIG instances can be run as confidential computing trusted execution environments (TEEs) for the first time.[9] On Hopper and later GPUs, switching MIG mode on no longer requires a GPU reset.[7]

Blackwell extended MIG to the B200, GB200 and B300 data center GPUs, which keep the seven-instance maximum, and to RTX PRO Blackwell professional GPUs, which support two or four instances. Together with the Thor iGPU, where graphics support is a technology preview, the RTX PRO Blackwell cards are the only MIG-capable GPUs whose profiles can run graphics APIs.[2][4]

"NVIDIA Multi-Instance GPU (MIG) Enables Elastic Computing", official video from NVIDIA, published May 14, 2020. Watch on YouTube.

How partitioning works

Slices, GPU instances and compute instances

MIG divides a GPU along two axes. A memory slice is the smallest unit of GPU memory, together with its memory controllers and cache, and is roughly one eighth of the GPU's memory capacity and bandwidth. An SM slice is roughly one seventh of the streaming multiprocessors (SMs) available when the GPU is in MIG mode. One memory slice plus one SM slice forms a GPU slice.[3]

A GPU instance (GI) combines some number of GPU slices with GPU engines such as copy engines and video decoders. Each GPU instance has dedicated memory that limits both its capacity and its bandwidth, which is how MIG provides memory quality of service. A GPU instance can be further divided into compute instances (CIs): each compute instance gets dedicated SMs, while all compute instances in the same GPU instance share its memory and engines.[3] By default a MIG device consists of one GPU instance holding one compute instance.[5]

Using NVIDIA's A100 40 GB example, the GPU behaves as eight 5 GB memory slices and seven SM slices. Pairing one memory slice with one SM slice yields a 1g.5gb instance; pairing four of each yields a 4g.20gb instance. That 4g.20gb instance can then be split into, for example, a one-slice compute instance named 1c.4g.20gb or a two-slice one named 2c.4g.20gb.[3] In NVIDIA's example listing for an A100 40 GB, the 1g.5gb profile has 14 SMs and the full 7g.40gb profile has 98, out of the 108 SMs on the A100.[7][8]

Profiles and naming

GPU instances cannot be any size. The driver offers a fixed list of GPU instance profiles, and an administrator can mix profiles on one GPU as long as enough free slices remain.[3] Profile names give the number of compute slices followed by the memory size, so 3g.20gb is a three-slice instance with 20 GB. The names scale with the memory of each product: the A100 80 GB uses 1g.10gb, 2g.20gb, 3g.40gb, 4g.40gb and 7g.80gb for the same shapes.[3][5] A +me suffix marks a profile that includes media extensions (video decode, JPEG and optical flow units), and on RTX PRO Blackwell cards +gfx marks profiles with graphics support.[3][4][6]

The A100 40 GB profiles show how the scheme works:[3][7]

ProfileMemory shareSM shareSMsMax instancesVideo decoders (NVDEC)
1g.5gb1/81/71470
1g.5gb+me1/81/71411
1g.10gb2/81/71441
2g.10gb2/82/72831
3g.20gb4/83/74222
4g.20gb4/84/75612
7g.40gbFull7/79815

Expanded article table

Profiles also have fixed physical placements on the GPU. Because instances occupy specific positions, creating and destroying instances in different places can fragment the GPU, and where one instance sits determines which others can be created next to it.[3] On the A100, a combination of a four-slice and a three-slice instance (4 memory slices with 4 compute slices, and 4 memory slices with 3 compute slices) was not allowed before driver release R510.[3]

The largest and smallest instance sizes on some supported products, as listed in NVIDIA's guide:[3][6]

GPUSmallest profileFull-GPU profileMax instances
A100 40 GB1g.5gb7g.40gb7
A100 80 GB1g.10gb7g.80gb7
H100 80 GB1g.10gb7g.80gb7
B200 180 GB1g.23gb7g.180gb7
GB200 186 GB1g.23gb7g.186gb7
B300 288 GB1g.34gb7g.269gb7
RTX PRO 6000 Blackwell 96 GB1g.24gb4g.96gb4

Expanded article table

Supported hardware

NVIDIA's guide describes MIG as supported on GPUs starting with the Ampere generation (compute capability 8.0 and higher) and lists the specific products below; systems built from them, such as DGX, DGX Station and HGX, are also supported.[2] As of the guide's November 2026 revision, the list and minimum drivers are:[2][4]

ProductArchitectureMemoryMax instancesMinimum driver
A100 (SXM4 and PCIe)Ampere40 GB or 80 GB7R450 (450.80.02)
A30Ampere24 GB4R460 (460.73.01)
H100 (SXM5 and PCIe)Hopper80 GB or 94 GB7R525 (525.85.12)
H100 on GH200Hopper96 GB7Not listed
H20Hopper96 GB7R535 (535.161.07)
H200 (SXM5 and NVL)Hopper141 GB7R550 (550.90.07)
B200Blackwell180 GB7R570 (570.133.20)
GB200Blackwell186 GB7R570 (570.133.20)
B300Blackwell288 GB7R580 (580.82.07)
GB300 (DGX Station)Blackwell252 GB7R595 (595.58.03)
RTX PRO 6000 Blackwell (Server, Workstation and Max-Q)Blackwell96 GB4R575 (575.51.03)
RTX PRO 5000 BlackwellBlackwell48 GB2R575 (575.57.08)
RTX PRO 4500 BlackwellBlackwell32 GB2R575 (575.64)
Thor iGPUBlackwellUnified with CPU2R595 (JetPack 7.2)

Expanded article table

The Thor iGPU has no dedicated video memory and allows at most two instances at once, one for compute and one with graphics.[2][6] Related product pages include the H200, H20, GH200, B200, Blackwell Ultra and RTX PRO 6000.

Comparison with other GPU sharing methods

CUDA already offered two ways to run work concurrently on one GPU before MIG. CUDA streams let a single process submit work to independent queues, but everything (address space, SMs, memory bandwidth, caches and capacity) is shared, and an error affects the whole process. The Multi-Process Service (MPS) lets several cooperating processes share the GPU and can cap each client at a percentage of the SMs, but it offers no error isolation, and the scheduling hardware, memory bandwidth, caches and capacity remain shared.[3] NVIDIA's guide compares the three this way:[3]

CUDA streamsMPSMIG
Partition typeSingle processLogicalPhysical
Max partitionsUnlimited487
SM performance isolationNoYes (by percentage)Yes
Memory protectionNoYesYes
Memory bandwidth QoSNoNoYes
Error isolationNoNoYes
ReconfigurationDynamicAt process launchWhen idle

Expanded article table

Streams and MPS still work inside a MIG instance. MPS can run on top of MIG, with its 48-client maximum reduced in proportion to the compute instance size.[3][4]

Setup and management

MIG is part of the NVIDIA GPU driver and runs only on Linux distributions that CUDA supports.[4] It can be controlled with the nvidia-smi command line tool or programmatically through the NVIDIA Management Library (NVML).[7] An administrator first enables MIG mode on a GPU (for example nvidia-smi -i 0 -mig 1), then creates GPU instances by profile ID or name and the matching compute instances; enabling MIG mode alone does not let CUDA workloads run.[7] Toggling MIG mode requires the CAP_SYS_ADMIN capability, while creating and destroying instances requires a separate mig-config capability that root holds by default and that can be delegated to a group.[4] Processes that hold the driver open, such as the DCGM monitoring service on DGX systems, have to be stopped before MIG mode can be enabled.[4][7]

Behaviour differs between generations. On Ampere GPUs, enabling MIG mode makes the driver reset the GPU, and the setting is stored in the GPU's InfoROM, so it survives reboots until it is turned off. On Hopper and later GPUs no reset is needed, but MIG mode lasts only while the driver's kernel modules are loaded.[7] On every generation the instances themselves disappear on reboot or GPU reset, and NVIDIA points administrators to its MIG Partition Editor (mig-parted) tool, which can recreate a chosen layout at system startup.[7] mig-parted is an Apache-2.0 open-source tool that lets an administrator declare a set of possible MIG configurations for all GPUs on a node and then apply one of them.[11]

Applications select a MIG device through CUDA_VISIBLE_DEVICES, which accepts MIG device UUIDs. With driver R570 and later, one CUDA process can see several GPU instances but only one compute instance in each, and CUDA supports at most 64 MIG instances across all GPUs.[5][7] NVIDIA recommends its Data Center GPU Manager (DCGM) for monitoring MIG devices, including per-instance utilization metrics.[7]

The guide lists several restrictions in MIG mode:[4]

  • Graphics APIs are not supported, except on +gfx profiles of RTX PRO Blackwell GPUs and the Thor iGPU (a technology preview on Thor), where OpenGL and Vulkan work but instances have no display output.
  • Peer-to-peer transfers are supported only between MIG instances on the same GPU (with driver R570).
  • CUDA IPC works across compute instances but not across GPU instances.
  • GPUDirect RDMA is supported from GPU instances.
  • NCCL 2.31 and later have experimental MIG support, with each rank on its own instance.
  • Profiling of shared GPU resources is not supported.

Containers, Kubernetes and virtualization

MIG supports three deployment models: bare metal (including containers), GPU pass-through to Linux virtual machines, and NVIDIA vGPU on supported hypervisors. With vGPU, several virtual machines can share one MIG-capable GPU in parallel while keeping vGPU's isolation guarantees.[1][4] NVIDIA documents host provisioning of MIG for vGPU in its NVIDIA AI Enterprise documentation.[22] NVIDIA's documentation names Red Hat Virtualization and VMware vSphere among supported hypervisors.[1]

For containers, NVIDIA lists minimum versions of the NVIDIA Container Toolkit (1.3.0), the Kubernetes device plugin (0.7.0) and gpu-feature-discovery (0.2.0).[7] The NVIDIA GPU Operator for Kubernetes offers two MIG strategies. Under the single strategy every MIG device is advertised as an ordinary nvidia.com/gpu resource, which requires that if any GPU on a node is in MIG mode, every GPU on that node is in MIG mode, has at least one MIG device and uses the same MIG profile. Under the mixed strategy each MIG device is advertised as a resource named after its profile, such as nvidia.com/mig-1g.10gb.[10] The operator's MIG Manager watches the nvidia.com/mig.config node label, and when the label changes it stops the GPU pods, applies the new layout with mig-parted, and requires that no user workloads are running on the GPUs being reconfigured.[10]

Cloud platforms expose MIG as well. Google Cloud's Kubernetes Engine (GKE) lets a node pool partition the GPUs it supports for MIG (the GB200, B200, H200, H100, A100 and RTX PRO 6000) into up to seven slices. In Standard clusters each partition size needs its own node pool, and a pod can use at most one MIG instance. Google's documentation says it does not recommend MIG partitions in GKE for untrusted workloads and notes that GPU utilization metrics such as duty_cycle are not available for them.[12]

Research and limitations

MIG has attracted a body of systems research, much of it on the problem of choosing partitions. Cheng Tan and colleagues showed in 2021 that picking the most efficient MIG partitions for serving deep neural networks is NP-hard, defined it as a "Reconfigurable Machine Scheduling Problem," and reported that their MIG-serving system on Kubernetes could save up to 40% of GPUs at the same throughput compared with using the A100 as is.[13] MISO, presented at the 2022 ACM Symposium on Cloud Computing, uses MPS to estimate the best MIG partition for co-located jobs without paying the cost of reconfiguring MIG while it searches; its authors reported 49% and 16% lower average job completion time than an unpartitioned GPU and the optimal static partition scheme, respectively.[14] MIGPerf (2023) is an open-source benchmark for training and inference on MIG instances.[15]

Several studies point to costs of MIG's rigid, hardware-defined layout:

  • A 2026 evaluation in The Journal of Supercomputing found that MPS could improve performance by up to 30% and cut energy by about 20% in favourable cases but lost around 30% under memory contention, while MIG's isolation gave more consistent results with higher overhead and, in some cases, lower performance because of its rigid scheme.[16]
  • Hsu-Tzu Ting and co-authors noted that contention persists on shared components such as PCIe bandwidth and that the small number of valid configurations and rigid placement rules cause fragmentation as jobs arrive and leave; their online scheduler improved makespan by up to 35%.[17]
  • Flex-MIG (2025) argued that MIG's one-instance-per-job allocation model causes cluster-wide fragmentation and proposed a software layer that lets one job span several instances, improving makespan by up to 17%.[18]
  • Researchers studying power accounting found that GPUs lack hardware support for dividing power consumption among MIG instances and built software models to estimate it, with fair carbon reporting for tenants as a motivation.[19]
  • Work on placing large language model inference jobs on MIG GPUs reported up to a 2.85x improvement in the number of GPUs used and up to a 70% reduction in GPU wastage compared with baseline heuristics.[20]

Comparable features

Other accelerator vendors partition GPUs in different ways. AMD's Instinct MI300X, for example, has compute partitioning modes ranging from SPX, which presents the whole GPU as one device, to CPX, which presents each of its eight XCDs as a separate logical GPU, plus memory partitioning modes such as NPS1 (one 192 GB pool) and NPS4 (four 48 GB partitions).[21]

Continue reading

More from See also

More to read

Topics:AI HardwareAI InferenceAI InfrastructureNVIDIA

References

  1. ^1 ^2 ^3 ^4 ^5 ^6NVIDIA. "Introduction". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...introduction
  2. ^1 ^2 ^3 ^4 ^5 ^6NVIDIA. "Supported GPUs". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...supported-gpus
  3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13NVIDIA. "Concepts". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...concepts
  4. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9NVIDIA. "Deployment Considerations". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...deployment-considerations
  5. ^1 ^2 ^3NVIDIA. "MIG Device Names". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...mig-device-names
  6. ^1 ^2 ^3NVIDIA. "Supported MIG Profiles". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...supported-mig-profiles
  7. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12NVIDIA. "Getting Started with MIG". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...getting-started-with-mig
  8. ^1 ^2 ^3 ^4 ^5 ^6 ^7Krashinsky, Ronny; Giroux, Olivier; Jones, Stephen; Stam, Nick; Ramaswamy, Sridhar. "NVIDIA Ampere Architecture In-Depth". NVIDIA Technical Blog, May 14, 2020. developer.nvidia.com/...pere-architecture-in-depth
  9. ^1 ^2Andersch, Michael; Palmer, Greg; Krashinsky, Ronny; Stam, Nick; Mehta, Vishal; Brito, Gonzalo; Ramaswamy, Sridhar. "NVIDIA Hopper Architecture In-Depth". NVIDIA Technical Blog, March 22, 2022. developer.nvidia.com/...pper-architecture-in-depth
  10. ^1 ^2NVIDIA. "GPU Operator with MIG". NVIDIA GPU Operator documentation, accessed November 2026. docs.nvidia.com/...gpu-operator-mig
  11. ^NVIDIA. "mig-parted: MIG Partition Editor for NVIDIA GPUs". GitHub, accessed November 2026. github.com/...mig-parted
  12. ^Google Cloud. "Running multi-instance GPUs". GKE AI/ML, Google Cloud Documentation, last updated November 10, 2026. docs.cloud.google.com/...gpus-multi
  13. ^Tan, Cheng; Li, Zhichao; Zhang, Jian; Cao, Yu; Qi, Sikai; Liu, Zherui; Zhu, Yibo; Guo, Chuanxiong. "Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem". arXiv:2109.11067, September 2021. arxiv.org/...2109.11067
  14. ^Li, Baolin; Patel, Tirthak; Samsi, Siddharth; Gadepally, Vijay; Tiwari, Devesh. "MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant GPU Clusters". Proceedings of the 13th Symposium on Cloud Computing (SoCC '22), November 2022 (arXiv:2207.11428). doi.org/...3542929.3563510
  15. ^Zhang, Huaizheng; Li, Yuanming; Xiao, Wencong; Huang, Yizheng; Di, Xing; et al. "MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs". arXiv:2301.00407, January 2023. arxiv.org/...2301.00407
  16. ^Villarrubia, Jorge; Costero, Luis; Igual, Francisco D.; Olcoz, Katzalin. "A comprehensive evaluation of spatial co-execution on GPUs using MPS and MIG technologies". The Journal of Supercomputing 82(4), 175, 2026 (arXiv:2604.22430). arxiv.org/...2604.22430
  17. ^Ting, Hsu-Tzu; Chou, Jerry; Chen, Ming-Hung; Chung, I-Hsin. "An Online Fragmentation-Aware Scheduler for Managing GPU-Sharing Workloads on Multi-Instance GPUs". arXiv:2512.16099, December 2025. arxiv.org/...2512.16099
  18. ^Kim, Myeongsu; Yeom, Ikjun; Kim, Younghoon. "Flex-MIG: Enabling Distributed Execution on MIG". arXiv:2511.09143, November 2025. arxiv.org/...2511.09143
  19. ^Vamja, Tirth; Ray, Kaustabha; George, Felix; Devi, UmaMaheswari C. "On the Partitioning of GPU Power among Multi-Instances". arXiv:2501.17752, January 2025. arxiv.org/...2501.17752
  20. ^Turkkan, Bekir; Murali, Pavankumar; Harsha, Pavithra; Arora, Rohan; Vanloo, Gerard; Narayanaswami, Chandra. "Optimal Workload Placement on Multi-Instance GPUs". arXiv:2409.06646, September 2024. arxiv.org/...2409.06646
  21. ^AMD. "AMD Instinct MI300X GPU Partitioning Overview". AMD Instinct documentation, accessed November 2026. instinct.docs.amd.com/...overview
  22. ^NVIDIA. "Virtualization". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...virtualization

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

2 revisions · v2 · 3,254 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent fact-check before publication on November 18, 2026: 78 claims checked against 31 cited and independent sources; 0 material and 4 minor corrections applied before publication.

Cite this page: AI Wiki. "Multi-Instance GPU." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/mig

Edit article