Unleashing Next-Gen AI & HPC Performance with AMD ROCm™ 6.2

In the fast-paced world of AI models and high-performance computing (HPC) development, staying ahead of the curve is crucial. With the latest release of AMD ROCm™ 6.2, engineers and developers are equipped with groundbreaking tools and enhancements that promise to revolutionize their workflows. Whether you’re crafting cutting-edge AI applications or optimizing complex simulations, the new ROCm 6.2 offers unparalleled performance, efficiency, and scalability.

AMD unleashes next-gen AI & HPC performance with the latest release of AMD ROCm 6.2

Let’s dive into the top five key enhancements that make this release a game-changer for AI and HPC development.

• Extending vLLM Support in ROCm 6.2

The latest ROCm 6.2 release sees AMD expanding vLLM support, significantly advancing the AI inference capabilities of AMD Instinct™ Accelerators. Designed specifically for Large Language Models (LLMs), vLLM addresses critical inferencing challenges, such as efficient multi-GPU computation, reduced memory usage, and minimized computational bottlenecks.

With features like multi-GPU execution and FP8 KV cache, developers can now tackle these challenges head-on. The ROCm/vLLM branch even offers advanced experimental capabilities like FP8 GEMMs and custom decode paged attention. Integrating these features into AI pipelines promises improved performance and efficiency, making ROCm 6.2 a must-have for both existing and new AMD Instinct™ customers.

• Bitsandbytes Quantization Support

AMD ROCm now supports the Bitsandbytes quantization library, revolutionizing AI development by significantly enhancing memory efficiency and performance on AMD Instinct™ GPU accelerators. By utilizing 8-bit optimizers, Bitsandbytes can reduce memory usage during AI training, allowing developers to work with larger models on limited hardware.

Additionally, LLM.Int8() quantization optimizes AI, enabling effective deployment of LLMs on systems with less memory. The result is faster AI training and inference, improved overall efficiency, and broadened access to advanced AI capabilities. Integrating Bitsandbytes with ROCm is straightforward, providing developers with a cost-effective and scalable solution for AI model training and inference.

• ROCm Offline Installer Creator

The new ROCm Offline Installer Creator simplifies the installation process for systems without internet access or local repository mirrors. By creating a single installer file that includes all necessary dependencies, this tool provides a seamless deployment experience with a user-friendly GUI.

It integrates multiple installation tools into one unified interface, automating post-installation tasks like user group management and driver handling, ensuring correct and consistent installations. This is particularly beneficial for IT administrators, making the deployment of ROCm across various environments more efficient and error-free.

• Omnitrace and Omniperf Profiler Tools (Beta)

The introduction of Omnitrace and Omniperf Profiler Tools in ROCm 6.2 is set to transform AI and HPC development. Omnitrace offers a comprehensive view of system performance across CPUs, GPUs, NICs, and network fabrics, helping developers identify and address bottlenecks. Omniperf, on the other hand, provides detailed GPU kernel analysis for fine-tuning performance.

Together, these tools optimize both application-wide and compute-kernel-specific performance, supporting real-time performance monitoring. This enables developers to make informed decisions and adjustments throughout the development process, ensuring efficient resource utilization and faster AI training, inference, and HPC simulations.

• Broader FP8 Support

ROCm 6.2 has expanded FP8 support across its ecosystem, significantly enhancing the process of running AI models, particularly in inferencing. FP8 support addresses key challenges such as memory bottlenecks and high latency associated with higher precision formats. By enabling larger models or batches to be handled within the same hardware constraints, FP8 support allows for more efficient training and inference processes. Additionally, reduced precision calculations in FP8 decrease latency involved in data transfers and computations. This expanded support includes:

• FP8 GEMM support in PyTorch and JAX via HipBLASLt
• XLA FP8 support in JAX and Flax
• vLLM optimization with FP8 capabilities
• FP8-specific collective operations in RCCL
• FP8-based Fused Flash attention in MIOPEN
• Standardized FP8 headers across libraries

With ROCm 6.2, AMD continues to demonstrate its commitment to providing robust, competitive, and innovative solutions for the AI and HPC community. This release equips developers with the tools and support needed to push the boundaries of what’s possible, fostering confidence in ROCm as the open platform of choice for next-generation computational tasks. Embrace these advancements and elevate your projects to unprecedented levels of performance and efficiency.

Discover the full range of new features introduced in ROCm 6.2 by reviewing the release notes.