# AMD Instinct MI350P

> Source: https://aiwiki.ai/wiki/amd_instinct_mi350p
> Updated: 2026-07-27
> Categories: AI Hardware, AI Inference
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution to "AI Wiki (aiwiki.ai)".

The AMD Instinct MI350P is a [PCIe](/wiki/pcie) add-in card built on [AMD](/wiki/amd)'s CDNA 4 architecture, introduced on May 7, 2026 as the third member of the Instinct MI350 series and the first AMD Instinct product in a conventional PCIe card form factor since the MI210 in 2022.[1][3][4] Where the rest of the series ships as OAM modules destined for dense, often liquid-cooled compute trays, the MI350P is a full-height, full-length, dual-slot, passively cooled board designed to drop into mainstream air-cooled enterprise servers. It carries 128 compute units, 144 GB of [HBM3E](/wiki/hbm3e) at 4 TB/s, and a 600 W typical board power that can be configured down to 450 W.[1][2] Those figures are almost exactly half of the OAM-module Instinct MI350X, the part that shares its CDNA 4 design with the [AMD Instinct MI355X](/wiki/amd_instinct_mi355x), and that is by design: AMD built a smaller chip out of the same silicon, one I/O die carrying four accelerator complex dies rather than two dies carrying eight.[4]

AMD returned to the card at its Advancing AI 2026 event on July 23, 2026, where the press release described it as bringing "seamless AI acceleration to existing infrastructure with leadership token economics" and attached a new claim: up to 4.2 times more tokens per second per dollar than the competition.[8] The wording ("AMD also launched the Instinct MI350P GPU") reads as a first announcement, but by that date the card had already been public for eleven weeks and had appeared in vendor systems at Dell Technologies World, Computex 2026, and HPE Discover 2026.[5][8]

## Why a PCIe card

For roughly three years the volume of the AI accelerator market moved toward modular parts: OAM and SXM modules sold as eight-GPU baseboards, and more recently as whole racks. Neither AMD nor NVIDIA had shipped a current-generation, HBM-equipped server GPU on a PCIe card in that period. ServeTheHome characterized the result as a hole in the market: customers who need less than a full 11 kW compute node, or who run [data centers](/wiki/data_center) that cannot host one, were left choosing between workstation-derived cards and nothing.[4]

Both vendors had partially filled that gap with graphics-derived silicon, AMD with the Radeon AI Pro series and NVIDIA with the [RTX PRO 6000](/wiki/nvidia_rtx_pro_6000) Blackwell Server Edition. Those parts offer much of the same numeric-format support but not the memory capacity or bandwidth that HBM provides.[4][5] The MI350P is AMD's answer for buyers who want the flagship memory subsystem in a slot they already have.

The retrofit argument is the product's entire reason to exist, and AMD's marketing is unusually direct about it. The brochure headline is "Built for the infrastructure you have," and the body copy promises no specialized cooling, no rack redesigns, and up to eight cards per server.[2] The 450 W configurable mode exists for the same reason: ServeTheHome noted that HPE was specifically highlighting the lower power option on its show-floor tags, because 450 W is what makes the card integrable into standard high-volume compute servers rather than dedicated GPU chassis.[5] This is the opposite end of AMD's portfolio from [Helios](/wiki/amd_helios_rack), the rack-scale system built around MI400-series GPUs, liquid cooling, and a proprietary scale-up fabric. Helios asks the buyer to build a new kind of data center; the MI350P asks the buyer to open a server.

## Half an MI350X, built on purpose

The MI350X and MI355X packages consist of two I/O dies, each carrying four accelerator complex dies (XCDs) with 32 compute units apiece, for 256 compute units and eight HBM3E stacks. The MI350P is one I/O die with four XCDs and four HBM3E stacks.[2][4]

ServeTheHome's Ryan Smith initially assumed the card used salvaged MI350X packages that had failed binning. AMD's technical briefing showed otherwise: the MI350P uses a purpose-built smaller chip that reuses the same chiplets in a smaller assembly.[4] AMD's published transistor count of 73 billion, against 185 billion for the MI350X, indicates the halving is not literal at the die level.[1][13] Peak engine clock is identical between the two parts at 2.2 GHz, so essentially every published throughput figure for the MI350P is precisely half its OAM sibling's.[1][13]

Board power is the one specification that did not halve. The MI350X is rated at 1,000 W and the MI355X at 1,400 W; the MI350P is 600 W, which is the ceiling the PCIe CEM specification itself defines. AMD chose to run the card as close to that limit as the standard allows.[4][13][14] Power arrives through a single 12V-2x6 connector mounted at the front of the card, opposite the I/O bracket.[1][5]

## Specifications

All figures below come from AMD's product page and product brochure, and AMD labels the throughput numbers "estimated" peak.[1][2]

| Specification | Value |
|---|---|
| Architecture | AMD CDNA 4 |
| Lithography | TSMC 3nm and 6nm FinFET |
| Chiplet configuration | 1 I/O die, 4 XCDs |
| Transistor count | 73 billion |
| Compute units | 128 (32 per XCD) |
| Stream processors | 8,192 |
| Matrix cores | 512 |
| Peak engine clock | 2.2 GHz |
| Memory | 144 GB HBM3E |
| Memory interface | 4,096-bit |
| Peak memory bandwidth | 4 TB/s |
| Infinity Cache (last level) | 128 MB |
| Memory ECC | Full-chip |
| MXFP4 matrix (peak) | 4.6 PFLOPS |
| MXFP6 matrix (peak) | 4.6 PFLOPS |
| MXFP8 matrix (peak) | 2.3 PFLOPS |
| OCP-FP8 matrix (peak) | 2.3 PFLOPS dense, 4.6 PFLOPS with structured sparsity |
| INT8 matrix (peak) | 2.3 POPS dense, 4.6 POPS with structured sparsity |
| FP16 and BF16 matrix (peak) | 1.15 PFLOPS dense, 2.3 PFLOPS with structured sparsity |
| FP32 matrix and vector (peak) | 72 TFLOPS |
| FP64 matrix and vector (peak) | 36 TFLOPS |
| Form factor | FHFL dual-slot PCIe CEM card, 10.5 in (267 mm) |
| Host interface | PCIe 5.0 x16 (128 GB/s) |
| GPU-to-GPU interconnect | None; PCIe only |
| Cooling | Passive, air cooled by chassis airflow |
| External power | 12V-2x6 |
| Typical board power | 600 W maximum, configurable to 450 W |
| Partitioning | Up to 4 GPU partitions at 36 GB each |
| Video decode | 2 decoder groups (HEVC/H.265, AVC/H.264, VP9, AV1), 20-core JPEG/MJPEG codec |
| Operating system support | Linux x86-64 |

Note that AMD's own launch comparison table, reproduced by Tom's Hardware, lists the MI350P at "2.3 PFLOPS FP16" and "4.6 PFLOPS FP8."[6] Those are the structured-sparsity figures. The dense numbers on AMD's product page are half as large.[1] Mixing the two bases is a common source of confusion across MI350-series coverage, and ServeTheHome flagged it explicitly when building its own comparison charts.[5]

### Peak versus delivered

AMD did something with the MI350P that it did not do for the MI350X launch: it published typical delivered throughput alongside peak theoretical throughput.[4] The delivered figures are 2,299 TFLOPS at MXFP4, 1,529 TFLOPS at FP8, and 713 TFLOPS at BF16, against peaks of 4,600, 2,300, and 1,150 respectively.[3][7] AMD did not explain its methodology, and both compute throughput and memory bandwidth are shown as affected.[4] ServeTheHome called the disclosure welcome and noted that MXFP6 in particular falls well short of its theoretical peak.[4] AMD's own blog leads with the pair, describing "estimated 2,299 teraflops (TFLOPS) and up to 4,600 peak TFLOPS at MXFP4."[3]

## The missing Infinity Fabric links

The MI350P's largest architectural compromise is that AMD does not expose any GPU-to-GPU Infinity Fabric links on it. The MI350X and MI355X each carry seven Infinity Fabric links at 153 GB/s peak per link; on the MI350P, multi-card communication runs over the PCIe 5.0 x16 bus and nothing else.[1][4][13][14]

That constrains what an eight-card server is good for. As ServeTheHome put it, such a system is a better fit for running eight models than for spreading one large model across eight GPUs, because tensor parallelism across a slow interconnect is expensive.[4] The practical ceiling for a single-card model is the 144 GB memory pool, which in FP4 or FP6 quantization still covers a large share of open-weight models. The absence of bridges has a compensating benefit in 2U chassis: Gigabyte's G294-Z22-AAP2 stacks GPUs in pairs on either side of a slim motherboard, a layout that bridge PCBs generally cannot span, so the MI350P loses nothing there that a bridged card would not also lose.[5]

Partitioning survives from the rest of the series in reduced form. With four XCDs instead of eight, the card supports up to four physical partitions in CPX mode, each one XCD with 36 GB of memory, the same per-partition arrangement as the MI350X.[2][4] AMD's product page lists SR-IOV support; the brochure describes it as "future SR-IOV support for up to four partitions," so the two documents do not agree on whether it was live at introduction.[1][2]

## Comparison within the MI350 series

| | MI350P | MI350X | MI355X |
|---|---|---|---|
| Form factor | PCIe add-in card | OAM module | OAM module |
| Compute units | 128 | 256 | 256 |
| Peak engine clock | 2.2 GHz | 2.2 GHz | 2.4 GHz |
| Memory | 144 GB HBM3E | 288 GB HBM3E | 288 GB HBM3E |
| Memory bandwidth | 4 TB/s | 8 TB/s | 8 TB/s |
| Last level cache | 128 MB | 256 MB | 256 MB |
| MXFP4 matrix (peak) | 4.6 PFLOPS | 9.2 PFLOPS | 10.1 PFLOPS |
| MXFP8 matrix (peak) | 2.3 PFLOPS | 4.6 PFLOPS | 5 PFLOPS |
| FP64 (peak) | 36 TFLOPS | 72.1 TFLOPS | 78.6 TFLOPS |
| Transistors | 73 billion | 185 billion | 185 billion |
| Infinity Fabric links | None | 7 | 7 |
| Typical board power | 600 W (450 W option) | 1,000 W | 1,400 W |
| Cooling | Passive, air | Passive OAM | Passive and active |
| Introduced | 2026-05-07 | 2025-06-12 | 2025-06-12 |

Sources: AMD product pages for all three parts.[1][13][14]

## Competitive position

AMD's chosen comparison point is NVIDIA's [H200 NVL](/wiki/nvidia_h200), the PCIe card version of the [Hopper](/wiki/nvidia_hopper) generation H200. That card carries 141 GB of HBM3E at 4.8 TB/s in a dual-slot air-cooled board with a configurable TDP up to 600 W, and supports 2-way or 4-way NVLink bridges at 900 GB/s per GPU.[12] So the H200 NVL has slightly less memory and more bandwidth than the MI350P, and unlike the MI350P it retains a real GPU-to-GPU interconnect.

Where the MI350P pulls ahead is architecture generation. Hopper predates the industry's move to FP6 and FP4, so NVIDIA publishes no FP6 or FP4 figures for the H200 NVL, while the MI350P supports MXFP4 and MXFP6 natively.[5][12] Tom's Hardware, comparing peak theoretical numbers, put the MI350P about 20 percent ahead on FP64, 43 percent ahead on FP16 and 39 percent ahead on FP8.[6] Check the bases before repeating those figures. The FP64 lead holds only against the H200 NVL's 30 TFLOPS vector rate; NVIDIA rates the same card at 60 TFLOPS on FP64 Tensor Core, comfortably above the MI350P's 36 TFLOPS matrix FP64.[12] The FP16 and FP8 gaps do compare like with like, since both vendors' headline numbers there are structured-sparsity figures, but measured against NVIDIA's published 1,671 TFLOPS FP16 and 3,341 TFLOPS FP8 the margins work out at about 38 percent in each case rather than 43 and 39.[1][12] All of this is paper arithmetic, not measured performance.

The [Blackwell](/wiki/nvidia_blackwell)-generation alternative in this slot is the RTX PRO 6000 Blackwell Server Edition, which uses 96 GB of GDDR7 rather than HBM and adds RT cores for mixed graphics and compute workloads.[5] All three cards are 600 W passively cooled dual-slot boards.[5] As of July 2026 NVIDIA had not released a Blackwell-generation PCIe card with HBM; ServeTheHome saw an Inventec chassis at Computex still listing an "8x B300 NVL" option that NVIDIA never shipped.[5] That absence is precisely the gap the MI350P targets, and it left AMD as the only vendor offering a current-generation server-grade accelerator on a PCIe card.[4]

## The 4.2x tokens per dollar claim

The single quantitative claim AMD attached to the MI350P at Advancing AI 2026 is that it delivers "up to 4.2x more tokens per second per dollar than the competition."[8] This is a vendor claim from AMD internal testing, not an independent measurement, and the supporting endnote is worth reading in full because it is doing more work than the headline suggests.

Endnote MI350P-007 states that the test is based on AMD internal testing in July 2026, comparing one MI350P against one NVIDIA H200 NVL on [Llama 3.3](/wiki/llama_3_3) 70B Instruct in FP8, measuring online serving output throughput per dollar in tokens per second per US dollar, at input and output sequence lengths of 1024/1024, across concurrency levels 1, 4, 8, 16, 32, 64, 128, 256 and 512, taking the median of three runs at each point. The stated result is the peak per-concurrency ratio, meaning the best single point on the curve rather than an average.[8]

AMD did publish prices, but not the ones the headline implies. The endnote gives an MI350P-based server at AMD internal estimated pricing of $327,238.40 and an RTX PRO 6000-based server at an OEM list price of $265,928.24 as of 2026-07-16. No price is given for an H200 NVL server anywhere in the endnote.[8] The detailed configuration block likewise describes only two systems: eight MI350P cards and eight RTX PRO 6000 Blackwell Server Edition cards, both hosted in a Dell PowerEdge XE7745 with dual EPYC 9455 processors. The AMD side ran [ROCm](/wiki/rocm) 7.14.0 with AMD Inference Microservices; the NVIDIA side ran NVIDIA NIM containers on CUDA 13.2.[8]

Two things follow. First, a per-dollar metric is part benchmark and part pricing model, and the pricing input on AMD's own side is an internal estimate rather than a published list price. Second, the one price pair AMD does publish is not the pair behind the headline: it puts an MI350P server roughly 23 percent above an RTX PRO 6000 server, while the 4.2x number is measured against the H200 NVL, whose price appears nowhere in the endnote. The published inputs are not sufficient to reconstruct the 4.2x claim.

## Software

The MI350P uses the gfx950 target, the same LLVM target as the rest of the MI350 series, and appears in the ROCm 7.14.0 compatibility matrix as a supported device.[8][11] AMD's product page lists HIP, OpenCL and OpenMP API support and framework support for [PyTorch](/wiki/pytorch), TensorFlow, JAX, ONNX Runtime, [SGLang](/wiki/sglang), Triton, Kokkos and RAJA, on Linux x86-64 only.[1]

AMD pairs the card with its Enterprise AI reference stack rather than selling software separately. That stack includes the AMD GPU Operator for Kubernetes lifecycle management, AMD Inference Microservices (AIMs) as prepackaged model-serving containers, AMD Resource Manager, and AMD AI Workbench. AMD provides it to partners at no licensing cost, and the open-source, no-license-fee framing is a recurring theme in AMD's positioning against bundled proprietary inference software.[2][3] The AIMs container is what AMD used for its own MI350P benchmark runs.[8]

## Systems, partners and deployments

AMD named Dell Technologies, HPE, Cisco, [Lenovo](/wiki/lenovo), Supermicro and Giga Computing (Gigabyte) as OEM partners at introduction, with supporting statements from each, plus software partners including Red Hat, Akamai, [Broadcom](/wiki/broadcom) (VMware Cloud Foundation), Nutanix, Uniphore, Kamiwaza and Seekr.[3]

Dell said on the day of introduction that its PowerEdge XE7745 and R7725, both standard air-cooled servers, would support the MI350P beginning in July 2026, with no data center redesign required.[9] By mid-July the card had appeared across the show circuit: eight cards inside a PowerEdge XE7745 at Dell Technologies World, an HPE ProLiant DL385 Gen11 at HPE Discover, and at Computex 2026 the Gigabyte G294-Z22-AAP2, the Inventec P5000AG7, the MiTAC TN85-B8261 and the ASUS ESC8000A-E13.[5]

The largest single configuration announced is from Liqid, unveiled at Advancing AI 2026 on July 23, 2026. The Liqid UltraStack 30 pools 30 MI350P cards behind dual-socket AMD EPYC 9005 processors using Liqid's PCIe-fabric GPU pooling, for roughly 4.3 TB of aggregate HBM3E and 69 PFLOPS of FP8 in about 22 kW.[10] Liqid's throughput and efficiency figures for the platform are company projections, not measured results.[10]

## Availability, pricing and independent testing

AMD did not publish a list price or a general availability date at introduction, and ServeTheHome's Ryan Smith confirmed in the comments under his own launch piece that no pricing or availability had been announced.[4] Availability in practice has come through OEM platforms rather than direct card sales, with Dell's July 2026 window the most specific commitment on record.[9]

As of 2026-07-27 there was no independent benchmarking of the MI350P. ServeTheHome, which had intended to test the card against the RTX PRO 6000 Blackwell Server Edition, reported on July 17, 2026 that despite AMD seeding a relatively high volume of sample cards, customer demand meant the outlet could not get access to one.[5] StorageReview likewise had not tested it.[7] Nor does the card appear in [MLPerf](/wiki/mlperf) Inference v6.0: those results were published on April 1, 2026, five weeks before the MI350P was introduced, and in that round AMD's own submissions all ran on MI355X systems while partner submissions covered the MI300X, MI325X, MI350X and MI355X.[15] Every performance figure in circulation as of July 2026 therefore traces to AMD or to its partners.

## Where it sits relative to the MI400 series

The MI350P is a CDNA 4 part and shares no silicon with the MI400 generation that AMD detailed at [Advancing AI 2026](/wiki/amd_advancing_ai_2026), where the [MI455X](/wiki/amd_instinct_mi455x) took the flagship training and cloud role and the MI430X targeted HPC and sovereign deployments with hardware FP64.[8] Those parts are OAM and rack-scale products aimed at operators building new [AI accelerator](/wiki/ai_accelerator) capacity. The MI350P sells against a different constraint entirely: an enterprise with existing racks, existing air cooling, and a power budget measured per server rather than per rack. AMD kept it in the July 2026 announcement precisely to mark that the [MI400](/wiki/amd_mi400) generation does not replace it.

## See also

- [AMD Instinct MI355X](/wiki/amd_instinct_mi355x)
- [AMD Instinct MI325X](/wiki/amd_instinct_mi325x)
- [AMD Instinct MI300X](/wiki/amd_instinct_mi300x)
- [AMD Helios](/wiki/amd_helios_rack)
- [NVIDIA H200](/wiki/nvidia_h200)
- [ROCm](/wiki/rocm)
- [HBM3E](/wiki/hbm3e)
- [FP4](/wiki/fp4)
- [Retrieval-augmented generation](/wiki/retrieval_augmented_generation)

## References

1. "AMD Instinct MI350P PCIe Cards." AMD, 2026. https://www.amd.com/en/products/accelerators/instinct/mi350/mi350p.html
2. "Product Brochure: AMD Instinct MI350P PCIe Card." AMD, 2026-05. https://www.amd.com/content/dam/amd/en/documents/epyc-business-docs/other/amd-instinct-mi350p-product-brochure.pdf
3. "AMD Instinct MI350P PCIe GPUs: Run Enterprise AI on Your Existing Infrastructure." AMD Blogs, 2026-05-07. https://www.amd.com/en/blogs/2026/amd-instinct-mi350p-pcie-gpus-run-enterprise-ai-on-your.html
4. Ryan Smith. "AMD Intros Instinct MI350P Accelerator: CDNA 4 Comes to PCIe Cards." ServeTheHome, 2026-05-07. https://www.servethehome.com/amd-intros-instinct-mi350p-accelerator-cdna-4-comes-to-pcie-cards/
5. Patrick Kennedy. "The AMD Instinct MI350P is a HBM PCIe AI Accelerator That Has Been All Over." ServeTheHome, 2026-07-17. https://www.servethehome.com/the-amd-instinct-mi350p-is-a-hbm-pcie-accelerator-that-has-been-all-over/
6. Aaron Klotz. "AMD announces MI350P PCIe AI accelerator card with 144GB of HBM3E." Tom's Hardware, 2026-05-07. https://www.tomshardware.com/pc-components/gpus/amd-announces-mi350p-pcie-ai-accelerator-card-with-144gb-of-hbm3e-roughly-40-percent-faster-in-fp16-and-fp8-theoretical-compute-compared-to-nvidias-h200-nvl-competitor
7. "AMD Instinct MI350P: Enterprise PCIe AI Inference Returns to Standard Servers." StorageReview, 2026-05-07. https://www.storagereview.com/news/amd-instinct-mi350p-enterprise-pcie-ai-inference-returns-to-standard-servers
8. "AAI 2026: AMD Delivers Full-Stack Compute for the Agentic AI Era." AMD via GlobeNewswire, 2026-07-23. https://www.globenewswire.com/news-release/2026/07/23/3332491/0/en/AAI-2026-AMD-Delivers-Full-Stack-Compute-for-the-Agentic-AI-Era.html
9. "Dell and AMD Are Expanding What's Possible for On-Premises AI." Dell Technologies Blog, 2026-05-07. https://www.dell.com/en-us/blog/dell-and-amd-are-expanding-what-s-possible-for-on-premises-ai/
10. "Liqid Debuts 30-GPU AI Platform Powered by AMD Instinct MI350P." Converge Digest, 2026-07-24. https://convergedigest.com/liqid-debuts-30-gpu-ai-platform-powered-by-amd-instinct-mi350p/
11. "ROCm 7.14.0 compatibility matrix." AMD ROCm Documentation, 2026. https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html
12. "NVIDIA H200 Tensor Core GPU." NVIDIA, 2026. https://www.nvidia.com/en-us/data-center/h200/
13. "AMD Instinct MI350X Accelerators." AMD, 2026. https://www.amd.com/en/products/accelerators/instinct/mi350/mi350x.html
14. "AMD Instinct MI355X Accelerators." AMD, 2026. https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html
15. "AMD Instinct GPUs MLPerf Inference v6.0 Submission." AMD ROCm Blogs, 2026-04-01. https://rocm.blogs.amd.com/artificial-intelligence/mlperf-inference-v6.0/README.html