NVIDIA is also revealing its next-generation Grace CPU design, which will be supported by two new Superchip platforms in addition to the Hopper GH100 GPU. The Grace CPU and Grace Hopper platforms are designed for HPC and AI ecosystems, respectively.

NVIDIA has designed a three-chip data centre strategy in which it intends to use the whole AI and data centre ecosystem using its Hopper GPU, Grace CPU, and a combination of both IPs. NVIDIA has two platforms in the works for non-traditional workloads. These will be known as ‘SUPERCHIPS,’ and will come in CPU-only and GPU+CPU configurations.

First up is NVIDIA’s Grace Hopper Superchip, a technology designed for massive-scale AI and high-performance computing. This Grace CPU and Hopper GPU combination has 600 GB of GPU memory, a 900 GB/s coherent NVLINK interface, and runs the NVIDIA Computing Stack software suite.

All of this provides a 30x system memory bandwidth to the GPU in servers using SUPERCHIP products, resulting in a significant performance boost. In the first half of 2023, the Grace Hopper Superchip platform is projected to be ready.

CPU+GPU designed for giant-scale AI and HPC
New 900 gigabytes per second (GB/s) coherent interface, 7X faster than PCIe Gen 5
30X higher aggregate system memory bandwidth to GPU compared to DGX A100
Runs all NVIDIA software stacks and platforms, including NVIDIA HPC, NVIDIA AI, and NVIDIA Omniverse

The Grace Hopper Superchip uses NVIDIA NVLink-C2C to merge the Grace and Hopper architectures to create a CPU+GPU coherent memory paradigm for accelerated AI and high-performance computing (HPC) workloads. The Grace CPU is the second Superchip platform, which includes high-performance Grace ARM CPU cores geared at HPC and AI infrastructure.

NVIDIA launches its Grace Hopper & Grace CPU Superchips with 600GB of GPU memory

The computing platform has 144 cores based on the ARM Neoverse architecture, with a performance of 740 SPECrate@2017 int base (estimated). In the same performance metric, previous estimates were 300, thus this is an almost 2.5x gain since it was first revealed.

NVLink-C2C technology is used by NVIDIA Grace CPU Superchip to deliver 144 Arm v9 cores and 1 TB/s memory bandwidth. It’s also the first platform to include ECC-enabled LPDRR5X RAM, with up to 1 TB/s of memory bandwidth.

High-performance CPU for HPC and cloud computing
Super chip design with up to 144 Arm v9 CPU cores
World’s first LPDDR5x with ECC Memory, 1TB/s total bandwidth
SPECrate2017_int_base over 740 (estimated)
900 GB/s coherent interface, 7X faster than PCIe Gen 5
2X the packaging density of DIMM-based solutions
2X the performance per watt of today’s leading CPU
Runs all NVIDIA software stacks and platforms, including RTX, HPC, AI, and Omniverse

In comparison to standard Intel and AMD servers, the platform runs on NVIDIA’s Computing Stack software package and promises 2x the performance per watt. The platform is planned to debut in the first half of 2023. The Grace Superchips are predicted to have a 500W power rating.

Grace, according to NVIDIA, is a highly specialised processor designed for workloads like training next-generation NLP models with over 1 trillion parameters. A Grace CPU-based system will give 10x quicker performance than today’s state-of-the-art NVIDIA DGX-based systems, which run on x86 CPUs when tightly paired with NVIDIA GPUs.

Grace was designed by NVIDIA by utilising Arm’s data centre architecture’s tremendous flexibility. NVIDIA is advancing the aim of technology variety in the AI and HPC sectors, where choice is critical to providing the innovation needed to tackle the world’s most pressing challenges, by introducing a new server-class CPU.