Multi-Instance GPU
Multi-Instance GPU (MIG) is a hardware partitioning feature of NVIDIA GPUs, starting with the Ampere-generation A100, that splits one physical GPU into as many as seven separate GPU instances, each with its own compute resources and its own share of the memory system.[1][8] NVIDIA describes the partitions as isolated: the on-chip crossbar ports, L2 cache banks, memory controllers and DRAM address buses are each assigned to a single instance, so a workload keeps the same cache allocation and memory bandwidth even when a neighbouring instance saturates its own.[1] The feature was aimed first at cloud providers and other multi-tenant operators who want to run several smaller jobs, such as low-batch inference, on one large accelerator without the jobs interfering with each other.[1][8]
NVIDIA introduced MIG with the A100 in May 2020, gave the H100 a second-generation version in 2022, and as of November 2026 supports it on Ampere, Hopper and Blackwell data center GPUs, on RTX PRO Blackwell professional cards and on the integrated GPU of Jetson Thor.[2][8][9] MIG is managed through the NVIDIA driver, the nvidia-smi tool and the NVIDIA Management Library, and it works with containers, Kubernetes and virtual machines.[1][7]
History
NVIDIA announced MIG on May 14, 2020 as a new feature of the A100 Tensor Core GPU, which is based on its Ampere architecture.[8] The launch description said an A100 could be "partitioned into as many as seven separate GPU Instances for CUDA applications," and pitched the feature mainly at cloud service providers, claiming they could offer "up to 7x more GPU Instances for no additional cost."[8] At launch, MIG worked under Linux, with containers through runtimes such as Docker Engine, and with hypervisors; NVIDIA said Kubernetes support was "coming soon."[8] The Ampere-based A30 also supports MIG, with up to four instances.[2]
The Hopper architecture, announced on March 22, 2022, brought what NVIDIA called second-generation MIG. According to NVIDIA, each H100 GPU instance has about three times the compute capacity and nearly twice the memory bandwidth of an A100 instance, each of the up to seven instances has dedicated NVDEC video decoders and NVJPG units, each instance has its own performance monitors, and MIG instances can be run as confidential computing trusted execution environments (TEEs) for the first time.[9] On Hopper and later GPUs, switching MIG mode on no longer requires a GPU reset.[7]
Blackwell extended MIG to the B200, GB200 and B300 data center GPUs, which keep the seven-instance maximum, and to RTX PRO Blackwell professional GPUs, which support two or four instances. Together with the Thor iGPU, where graphics support is a technology preview, the RTX PRO Blackwell cards are the only MIG-capable GPUs whose profiles can run graphics APIs.[2][4]
How partitioning works
Slices, GPU instances and compute instances
MIG divides a GPU along two axes. A memory slice is the smallest unit of GPU memory, together with its memory controllers and cache, and is roughly one eighth of the GPU's memory capacity and bandwidth. An SM slice is roughly one seventh of the streaming multiprocessors (SMs) available when the GPU is in MIG mode. One memory slice plus one SM slice forms a GPU slice.[3]
A GPU instance (GI) combines some number of GPU slices with GPU engines such as copy engines and video decoders. Each GPU instance has dedicated memory that limits both its capacity and its bandwidth, which is how MIG provides memory quality of service. A GPU instance can be further divided into compute instances (CIs): each compute instance gets dedicated SMs, while all compute instances in the same GPU instance share its memory and engines.[3] By default a MIG device consists of one GPU instance holding one compute instance.[5]
Using NVIDIA's A100 40 GB example, the GPU behaves as eight 5 GB memory slices and seven SM slices. Pairing one memory slice with one SM slice yields a 1g.5gb instance; pairing four of each yields a 4g.20gb instance. That 4g.20gb instance can then be split into, for example, a one-slice compute instance named 1c.4g.20gb or a two-slice one named 2c.4g.20gb.[3] In NVIDIA's example listing for an A100 40 GB, the 1g.5gb profile has 14 SMs and the full 7g.40gb profile has 98, out of the 108 SMs on the A100.[7][8]
Profiles and naming
GPU instances cannot be any size. The driver offers a fixed list of GPU instance profiles, and an administrator can mix profiles on one GPU as long as enough free slices remain.[3] Profile names give the number of compute slices followed by the memory size, so 3g.20gb is a three-slice instance with 20 GB. The names scale with the memory of each product: the A100 80 GB uses 1g.10gb, 2g.20gb, 3g.40gb, 4g.40gb and 7g.80gb for the same shapes.[3][5] A +me suffix marks a profile that includes media extensions (video decode, JPEG and optical flow units), and on RTX PRO Blackwell cards +gfx marks profiles with graphics support.[3][4][6]
The A100 40 GB profiles show how the scheme works:[3][7]
| Profile | Memory share | SM share | SMs | Max instances | Video decoders (NVDEC) |
|---|---|---|---|---|---|
| 1g.5gb | 1/8 | 1/7 | 14 | 7 | 0 |
| 1g.5gb+me | 1/8 | 1/7 | 14 | 1 | 1 |
| 1g.10gb | 2/8 | 1/7 | 14 | 4 | 1 |
| 2g.10gb | 2/8 | 2/7 | 28 | 3 | 1 |
| 3g.20gb | 4/8 | 3/7 | 42 | 2 | 2 |
| 4g.20gb | 4/8 | 4/7 | 56 | 1 | 2 |
| 7g.40gb | Full | 7/7 | 98 | 1 | 5 |
Profiles also have fixed physical placements on the GPU. Because instances occupy specific positions, creating and destroying instances in different places can fragment the GPU, and where one instance sits determines which others can be created next to it.[3] On the A100, a combination of a four-slice and a three-slice instance (4 memory slices with 4 compute slices, and 4 memory slices with 3 compute slices) was not allowed before driver release R510.[3]
The largest and smallest instance sizes on some supported products, as listed in NVIDIA's guide:[3][6]
| GPU | Smallest profile | Full-GPU profile | Max instances |
|---|---|---|---|
| A100 40 GB | 1g.5gb | 7g.40gb | 7 |
| A100 80 GB | 1g.10gb | 7g.80gb | 7 |
| H100 80 GB | 1g.10gb | 7g.80gb | 7 |
| B200 180 GB | 1g.23gb | 7g.180gb | 7 |
| GB200 186 GB | 1g.23gb | 7g.186gb | 7 |
| B300 288 GB | 1g.34gb | 7g.269gb | 7 |
| RTX PRO 6000 Blackwell 96 GB | 1g.24gb | 4g.96gb | 4 |
Supported hardware
NVIDIA's guide describes MIG as supported on GPUs starting with the Ampere generation (compute capability 8.0 and higher) and lists the specific products below; systems built from them, such as DGX, DGX Station and HGX, are also supported.[2] As of the guide's November 2026 revision, the list and minimum drivers are:[2][4]
| Product | Architecture | Memory | Max instances | Minimum driver |
|---|---|---|---|---|
| A100 (SXM4 and PCIe) | Ampere | 40 GB or 80 GB | 7 | R450 (450.80.02) |
| A30 | Ampere | 24 GB | 4 | R460 (460.73.01) |
| H100 (SXM5 and PCIe) | Hopper | 80 GB or 94 GB | 7 | R525 (525.85.12) |
| H100 on GH200 | Hopper | 96 GB | 7 | Not listed |
| H20 | Hopper | 96 GB | 7 | R535 (535.161.07) |
| H200 (SXM5 and NVL) | Hopper | 141 GB | 7 | R550 (550.90.07) |
| B200 | Blackwell | 180 GB | 7 | R570 (570.133.20) |
| GB200 | Blackwell | 186 GB | 7 | R570 (570.133.20) |
| B300 | Blackwell | 288 GB | 7 | R580 (580.82.07) |
| GB300 (DGX Station) | Blackwell | 252 GB | 7 | R595 (595.58.03) |
| RTX PRO 6000 Blackwell (Server, Workstation and Max-Q) | Blackwell | 96 GB | 4 | R575 (575.51.03) |
| RTX PRO 5000 Blackwell | Blackwell | 48 GB | 2 | R575 (575.57.08) |
| RTX PRO 4500 Blackwell | Blackwell | 32 GB | 2 | R575 (575.64) |
| Thor iGPU | Blackwell | Unified with CPU | 2 | R595 (JetPack 7.2) |
The Thor iGPU has no dedicated video memory and allows at most two instances at once, one for compute and one with graphics.[2][6] Related product pages include the H200, H20, GH200, B200, Blackwell Ultra and RTX PRO 6000.
Comparison with other GPU sharing methods
CUDA already offered two ways to run work concurrently on one GPU before MIG. CUDA streams let a single process submit work to independent queues, but everything (address space, SMs, memory bandwidth, caches and capacity) is shared, and an error affects the whole process. The Multi-Process Service (MPS) lets several cooperating processes share the GPU and can cap each client at a percentage of the SMs, but it offers no error isolation, and the scheduling hardware, memory bandwidth, caches and capacity remain shared.[3] NVIDIA's guide compares the three this way:[3]
| CUDA streams | MPS | MIG | |
|---|---|---|---|
| Partition type | Single process | Logical | Physical |
| Max partitions | Unlimited | 48 | 7 |
| SM performance isolation | No | Yes (by percentage) | Yes |
| Memory protection | No | Yes | Yes |
| Memory bandwidth QoS | No | No | Yes |
| Error isolation | No | No | Yes |
| Reconfiguration | Dynamic | At process launch | When idle |
Streams and MPS still work inside a MIG instance. MPS can run on top of MIG, with its 48-client maximum reduced in proportion to the compute instance size.[3][4]
Setup and management
MIG is part of the NVIDIA GPU driver and runs only on Linux distributions that CUDA supports.[4] It can be controlled with the nvidia-smi command line tool or programmatically through the NVIDIA Management Library (NVML).[7] An administrator first enables MIG mode on a GPU (for example nvidia-smi -i 0 -mig 1), then creates GPU instances by profile ID or name and the matching compute instances; enabling MIG mode alone does not let CUDA workloads run.[7] Toggling MIG mode requires the CAP_SYS_ADMIN capability, while creating and destroying instances requires a separate mig-config capability that root holds by default and that can be delegated to a group.[4] Processes that hold the driver open, such as the DCGM monitoring service on DGX systems, have to be stopped before MIG mode can be enabled.[4][7]
Behaviour differs between generations. On Ampere GPUs, enabling MIG mode makes the driver reset the GPU, and the setting is stored in the GPU's InfoROM, so it survives reboots until it is turned off. On Hopper and later GPUs no reset is needed, but MIG mode lasts only while the driver's kernel modules are loaded.[7] On every generation the instances themselves disappear on reboot or GPU reset, and NVIDIA points administrators to its MIG Partition Editor (mig-parted) tool, which can recreate a chosen layout at system startup.[7] mig-parted is an Apache-2.0 open-source tool that lets an administrator declare a set of possible MIG configurations for all GPUs on a node and then apply one of them.[11]
Applications select a MIG device through CUDA_VISIBLE_DEVICES, which accepts MIG device UUIDs. With driver R570 and later, one CUDA process can see several GPU instances but only one compute instance in each, and CUDA supports at most 64 MIG instances across all GPUs.[5][7] NVIDIA recommends its Data Center GPU Manager (DCGM) for monitoring MIG devices, including per-instance utilization metrics.[7]
The guide lists several restrictions in MIG mode:[4]
- Graphics APIs are not supported, except on
+gfxprofiles of RTX PRO Blackwell GPUs and the Thor iGPU (a technology preview on Thor), where OpenGL and Vulkan work but instances have no display output. - Peer-to-peer transfers are supported only between MIG instances on the same GPU (with driver R570).
- CUDA IPC works across compute instances but not across GPU instances.
- GPUDirect RDMA is supported from GPU instances.
- NCCL 2.31 and later have experimental MIG support, with each rank on its own instance.
- Profiling of shared GPU resources is not supported.
Containers, Kubernetes and virtualization
MIG supports three deployment models: bare metal (including containers), GPU pass-through to Linux virtual machines, and NVIDIA vGPU on supported hypervisors. With vGPU, several virtual machines can share one MIG-capable GPU in parallel while keeping vGPU's isolation guarantees.[1][4] NVIDIA documents host provisioning of MIG for vGPU in its NVIDIA AI Enterprise documentation.[22] NVIDIA's documentation names Red Hat Virtualization and VMware vSphere among supported hypervisors.[1]
For containers, NVIDIA lists minimum versions of the NVIDIA Container Toolkit (1.3.0), the Kubernetes device plugin (0.7.0) and gpu-feature-discovery (0.2.0).[7] The NVIDIA GPU Operator for Kubernetes offers two MIG strategies. Under the single strategy every MIG device is advertised as an ordinary nvidia.com/gpu resource, which requires that if any GPU on a node is in MIG mode, every GPU on that node is in MIG mode, has at least one MIG device and uses the same MIG profile. Under the mixed strategy each MIG device is advertised as a resource named after its profile, such as nvidia.com/mig-1g.10gb.[10] The operator's MIG Manager watches the nvidia.com/mig.config node label, and when the label changes it stops the GPU pods, applies the new layout with mig-parted, and requires that no user workloads are running on the GPUs being reconfigured.[10]
Cloud platforms expose MIG as well. Google Cloud's Kubernetes Engine (GKE) lets a node pool partition the GPUs it supports for MIG (the GB200, B200, H200, H100, A100 and RTX PRO 6000) into up to seven slices. In Standard clusters each partition size needs its own node pool, and a pod can use at most one MIG instance. Google's documentation says it does not recommend MIG partitions in GKE for untrusted workloads and notes that GPU utilization metrics such as duty_cycle are not available for them.[12]
Research and limitations
MIG has attracted a body of systems research, much of it on the problem of choosing partitions. Cheng Tan and colleagues showed in 2021 that picking the most efficient MIG partitions for serving deep neural networks is NP-hard, defined it as a "Reconfigurable Machine Scheduling Problem," and reported that their MIG-serving system on Kubernetes could save up to 40% of GPUs at the same throughput compared with using the A100 as is.[13] MISO, presented at the 2022 ACM Symposium on Cloud Computing, uses MPS to estimate the best MIG partition for co-located jobs without paying the cost of reconfiguring MIG while it searches; its authors reported 49% and 16% lower average job completion time than an unpartitioned GPU and the optimal static partition scheme, respectively.[14] MIGPerf (2023) is an open-source benchmark for training and inference on MIG instances.[15]
Several studies point to costs of MIG's rigid, hardware-defined layout:
- A 2026 evaluation in The Journal of Supercomputing found that MPS could improve performance by up to 30% and cut energy by about 20% in favourable cases but lost around 30% under memory contention, while MIG's isolation gave more consistent results with higher overhead and, in some cases, lower performance because of its rigid scheme.[16]
- Hsu-Tzu Ting and co-authors noted that contention persists on shared components such as PCIe bandwidth and that the small number of valid configurations and rigid placement rules cause fragmentation as jobs arrive and leave; their online scheduler improved makespan by up to 35%.[17]
- Flex-MIG (2025) argued that MIG's one-instance-per-job allocation model causes cluster-wide fragmentation and proposed a software layer that lets one job span several instances, improving makespan by up to 17%.[18]
- Researchers studying power accounting found that GPUs lack hardware support for dividing power consumption among MIG instances and built software models to estimate it, with fair carbon reporting for tenants as a motivation.[19]
- Work on placing large language model inference jobs on MIG GPUs reported up to a 2.85x improvement in the number of GPUs used and up to a 70% reduction in GPU wastage compared with baseline heuristics.[20]
Comparable features
Other accelerator vendors partition GPUs in different ways. AMD's Instinct MI300X, for example, has compute partitioning modes ranging from SPX, which presents the whole GPU as one device, to CPX, which presents each of its eight XCDs as a separate logical GPU, plus memory partitioning modes such as NPS1 (one 192 GB pool) and NPS4 (four 48 GB partitions).[21]
Continue reading
- NVIDIA A100The NVIDIA A100 Tensor Core GPU is a datacenter graphics processing unit that NVIDIA introduced on May 14, 2020 as the first product built on its Ampere architecture, and that became the dominant chip for…Mentioned 16 times above
- CUDACUDA (Compute Unified Device Architecture) is NVIDIA's platform and programming model for general-purpose computation on its graphics processing units.Mentioned 10 times above
- Ampere (microarchitecture)Ampere is a graphics processing unit (GPU) microarchitecture from Nvidia, announced on May 14, 2020, as the successor to the Volta and Turing architectures.Mentioned 8 times above
- NVIDIA HopperNVIDIA Hopper is the codename for NVIDIA's ninth-generation datacenter GPU microarchitecture, announced March 22, 2022 by CEO Jensen Huang at the GTC keynote.Mentioned 8 times above
- NVIDIA H100NVIDIA H100 (also called the H100 Tensor Core GPU) is a data-center graphics processing unit built by NVIDIA on the Hopper microarchitecture, fabricated with over 80 billion transistors on a custom TSMC 4N (4…Mentioned 6 times above
- ServingIn machine learning, serving (also called model serving or inference serving) is the process of deploying a trained model into a production environment so it can answer prediction requests, usually exposed as…Mentioned twice above
More from See also
More to read
References
- ^1 ^2 ^3 ^4 ^5 ^6NVIDIA. "Introduction". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...introduction
- ^1 ^2 ^3 ^4 ^5 ^6NVIDIA. "Supported GPUs". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...supported-gpus
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13NVIDIA. "Concepts". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...concepts
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9NVIDIA. "Deployment Considerations". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...deployment-considerations
- ^1 ^2 ^3NVIDIA. "MIG Device Names". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...mig-device-names
- ^1 ^2 ^3NVIDIA. "Supported MIG Profiles". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...supported-mig-profiles
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12NVIDIA. "Getting Started with MIG". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...getting-started-with-mig
- ^1 ^2 ^3 ^4 ^5 ^6 ^7Krashinsky, Ronny; Giroux, Olivier; Jones, Stephen; Stam, Nick; Ramaswamy, Sridhar. "NVIDIA Ampere Architecture In-Depth". NVIDIA Technical Blog, May 14, 2020. developer.nvidia.com/...pere-architecture-in-depth
- ^1 ^2Andersch, Michael; Palmer, Greg; Krashinsky, Ronny; Stam, Nick; Mehta, Vishal; Brito, Gonzalo; Ramaswamy, Sridhar. "NVIDIA Hopper Architecture In-Depth". NVIDIA Technical Blog, March 22, 2022. developer.nvidia.com/...pper-architecture-in-depth
- ^1 ^2NVIDIA. "GPU Operator with MIG". NVIDIA GPU Operator documentation, accessed November 2026. docs.nvidia.com/...gpu-operator-mig
- ^NVIDIA. "mig-parted: MIG Partition Editor for NVIDIA GPUs". GitHub, accessed November 2026. github.com/...mig-parted
- ^Google Cloud. "Running multi-instance GPUs". GKE AI/ML, Google Cloud Documentation, last updated November 10, 2026. docs.cloud.google.com/...gpus-multi
- ^Tan, Cheng; Li, Zhichao; Zhang, Jian; Cao, Yu; Qi, Sikai; Liu, Zherui; Zhu, Yibo; Guo, Chuanxiong. "Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem". arXiv:2109.11067, September 2021. arxiv.org/...2109.11067
- ^Li, Baolin; Patel, Tirthak; Samsi, Siddharth; Gadepally, Vijay; Tiwari, Devesh. "MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant GPU Clusters". Proceedings of the 13th Symposium on Cloud Computing (SoCC '22), November 2022 (arXiv:2207.11428). doi.org/...3542929.3563510
- ^Zhang, Huaizheng; Li, Yuanming; Xiao, Wencong; Huang, Yizheng; Di, Xing; et al. "MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs". arXiv:2301.00407, January 2023. arxiv.org/...2301.00407
- ^Villarrubia, Jorge; Costero, Luis; Igual, Francisco D.; Olcoz, Katzalin. "A comprehensive evaluation of spatial co-execution on GPUs using MPS and MIG technologies". The Journal of Supercomputing 82(4), 175, 2026 (arXiv:2604.22430). arxiv.org/...2604.22430
- ^Ting, Hsu-Tzu; Chou, Jerry; Chen, Ming-Hung; Chung, I-Hsin. "An Online Fragmentation-Aware Scheduler for Managing GPU-Sharing Workloads on Multi-Instance GPUs". arXiv:2512.16099, December 2025. arxiv.org/...2512.16099
- ^Kim, Myeongsu; Yeom, Ikjun; Kim, Younghoon. "Flex-MIG: Enabling Distributed Execution on MIG". arXiv:2511.09143, November 2025. arxiv.org/...2511.09143
- ^Vamja, Tirth; Ray, Kaustabha; George, Felix; Devi, UmaMaheswari C. "On the Partitioning of GPU Power among Multi-Instances". arXiv:2501.17752, January 2025. arxiv.org/...2501.17752
- ^Turkkan, Bekir; Murali, Pavankumar; Harsha, Pavithra; Arora, Rohan; Vanloo, Gerard; Narayanaswami, Chandra. "Optimal Workload Placement on Multi-Instance GPUs". arXiv:2409.06646, September 2024. arxiv.org/...2409.06646
- ^AMD. "AMD Instinct MI300X GPU Partitioning Overview". AMD Instinct documentation, accessed November 2026. instinct.docs.amd.com/...overview
- ^NVIDIA. "Virtualization". NVIDIA Multi-Instance GPU User Guide, accessed November 2026. docs.nvidia.com/...virtualization
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
2 revisions · v2 · 3,254 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check before publication on November 18, 2026: 78 claims checked against 31 cited and independent sources; 0 material and 4 minor corrections applied before publication.
Cite this page: AI Wiki. "Multi-Instance GPU." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/mig


