AMD Instinct MI300X GPUs and MI300A APUs based on CDNA3 launched

AMD has today unveiled the availability of its latest AMD Instinct™ MI300X accelerators, which provide industry-leading memory bandwidth for generative AI and superior performance for large language model (LLM) training and inferencing. The company also launched the AMD Instinct™ MI300A accelerated processing unit (APU), which integrates the latest AMD CDNA™ 3 architecture and “Zen 4” CPUs to offer groundbreaking performance for HPC and AI workloads.

Victor Peng, President of AMD, stated that the AMD Instinct MI300 Series accelerators have been engineered with their most advanced technologies, offering leading performance. He added that these accelerators are set to be deployed in large-scale cloud and enterprise environments. The company’s leadership hardware, software, and open ecosystem approach have enabled cloud providers, OEMs, and ODMs to introduce technologies that aid enterprises in adopting and deploying AI-powered solutions.

AMD Delivers Leadership Portfolio of Data Center AI Solutions with AMD Instinct MI300 Series 

Among the customers leveraging the latest AMD Instinct accelerator portfolio is Microsoft, which recently launched the new Azure ND MI300x v5 Virtual Machine (VM) series. These VMs are optimized for AI workloads and powered by AMD Instinct MI300X accelerators.

Moreover, El Capitan, a supercomputer equipped with AMD Instinct MI300A APUs and located at Lawrence Livermore National Laboratory, is anticipated to be the second exascale-class supercomputer powered by AMD. It is expected to deliver more than two exaflops of double precision performance once fully deployed. Oracle Cloud Infrastructure also plans to add AMD Instinct MI300X-based bare metal instances to its high-performance accelerated computing instances for AI.

In conjunction with the AMD Advancing AI event, several major OEMs showcased accelerated computing systems. Dell displayed the Dell PowerEdge XE9680 server, which features eight AMD Instinct MI300 Series accelerators and the new Dell Validated Design for Generative AI with AMD ROCm-p

owered AI frameworks. HPE announced the HPE Cray Supercomputing EX255a, the first supercomputing accelerator blade powered by AMD Instinct MI300A APUs, set to be available in early 2024. Lenovo also revealed its design support for the new AMD Instinct MI300 Series accelerators, with planned availability in the first half of 2024. Supermicro announced new additions to its H13 generation of accelerated servers powered by 4th Gen AMD EPYC™ CPUs and AMD Instinct MI300 Series accelerators.

AMD Instinct MI300X

• AMD Instinct MI300X accelerators are powered by the new AMD CDNA 3 architecture.
• Compared to the previous generation AMD Instinct MI250X, the MI300X offers nearly 40% more compute units, 1.5x more memory capacity, and 1.7x more peak theoretical memory bandwidth.
• These accelerators also support new math formats such as FP8 and sparsity, which are particularly useful for AI and HPC workloads.
• Instinct MI300X accelerators feature a best-in-class 192 GB of HBM3 memory capacity and 5.3 TB/s peak memory bandwidth for handling demanding AI workloads.
• The AMD Instinct Platform, built on an industry-standard OCP design with eight MI300X accelerators, offers an industry-leading 1.5TB of HBM3 memory capacity.
• The AMD Instinct Platform can offer a throughput increase of up to 1.6x when running inference on LLMs like BLOOM 176B4 compared to the Nvidia H100 HGX.

AMD Instinct MI300A

• The AMD Instinct MI300A APUs, which are the world’s first data centre APUs for HPC and AI, leverage 3D packaging and the 4th Gen AMD Infinity Architecture to deliver leadership performance on critical workloads.
• MI300A APUs combine high-performance AMD CDNA 3 GPU cores, the latest AMD “Zen 4” x86-based CPU cores, and 128GB of next-generation HBM3 memory.
• They deliver ~1.9x the performance per watt on FP32 HPC and AI workloads compared to the previous-gen AMD Instinct MI250X.
• AMD Instinct MI300A APUs benefit from integrating CPU and GPU cores on a single package, delivering a highly efficient platform along with the compute performance to accelerate the training the latest AI models.
• The APUs feature unified memory and cache resources, providing an easily programmable GPU platform, high-performance computing, fast AI training, and impressive energy efficiency to power the most demanding HPC and AI workloads.

AMD ROCm Software and Ecosystem Partners

AMD has announced the launch of its ROCm™ 6 open software platform, demonstrating its commitment to contributing advanced libraries to the open-source community and promoting open-source AI software development. The ROCm 6 software is a significant upgrade in AMD’s software tools, offering an approximately 8x increase in AI acceleration performance when used on MI300 Series accelerators for Llama 2 text generation, compared to the previous generation hardware and software.

Additionally, ROCm 6 introduces support for several key features crucial for generative AI, including FlashAttention, HIPGraph, and vLLM, amongst others. This positions AMD to utilize widely adopted open-source AI software models, algorithms, and frameworks like Hugging Face, PyTorch, and TensorFlow. This approach drives innovation, simplifies the deployment of AMD AI solutions, and unlocks the true potential of generative AI.

Furthermore, AMD continues to enhance its software capabilities through the acquisition of Nod.AI and Mipsology and strategic ecosystem partnerships such as Lamini and MosaicML. These collaborations enable the running of Large Language Models (LLMs) for enterprise customers and the use of AMD ROCm for LLM training on AMD Instinct accelerators with zero code changes.