More watts and more efficiency - Nvidia showcases their Hopper H100
Published: 22nd March 2022 | Source: Nvidia | Author: Mark Campbell
Nvidia's offering its HPC customers a HUGE upgrade with their Hopper H100 graphics chips
Nvidia has just revealed their H100 GPU, a new chip that utilises the company's new Hopper architecture. With Hopper, Nvidia has delivering a huge performance leap over Ampere, especially within the field of AI. When compared to their two-year-old Ampere A100, Nvidia's Hopper H100 will provide users with a huge performance leap.
With Hopper, Nvidia's HPC graphics architecture is more powerful and power hungry than ever before. Within the SMX form factor, Nvidia's Hopper H100 can consume up to 700 watts of power, 300 wats more than the company's highest power A100. That said, Nvidia has delivered its customers an efficiency boost despite this increased power budget, highlighting how much performance Nvidia now offers with Hopper. Within the PCIe form factor, Hopper will be limited to a lower TDP, but even with a lower power budget, Hopper is impressive.
With up to 80GB of HBM3 memory in its SMX configuration, Nvidia's Hopper H100 chip can offer users 3 TB/s of total memory bandwidth, and with NVLink 4, Nvidia's H100 will have 900GB/s of inter-chip bandwidth. Hopper also supports PCIe Gen 5, offering users 128 GB/s of bandwidth to users.
Ampere is Nvidia's most successful server/HPC graphics architecture to date, and with Hopper, Nvidia is aiming for even greater heights. With Hopper, Nvidia uses TSMC's 4nm lithography technology to deliver a chip with over 80 billion transistors.
Like Ampere, Hopper will launch in a cut-down configuration. While the chip supports six HBM memory chips, only five will be enabled. Additionally, there will be CUDA cores and Tensor cores that are disabled. These changes are made for yield reasons, as Nvidia's Hopper H100 chip is huge, and Nvidia wants to sell as many of these chips as possible to customers. Most of the chip is enabled, but Hopper has some untapped potential.
In theory, this means that Nvidia could release more powerful Hopper H100 based GPUs at a later date. Below are the specifications of Nvidia's Hopper H100 accelerator chips. Note that the specifications are different depending on how the chip is configured, with the chip's PCIe version being less powerful (and power-hungry).
Below, we are using Nvidia's non-sparse numbers for Tensor core computational performance, which are half of the performance that Hopper can deliver without sparsity enabled. This gives us a more apples-to-apples comparison with Nvidia's earlier HPC architectures.
| H100 (SMX) | H100 (PCIe) | A100 (SMX 80GB) | |
| Architecture | Hopper | Hopper | Ampere |
| Lithography Node | TSMC 4nm | TSMC 4nm | TSMC 7nm |
| Memory | 80GB | 80GB | 80GB |
| Memory Speed | 4.8 Gbps HBM3 | 3.2 Gbps HBM2e | 3.2 Gbps HBM2e |
| Memory Bus | 5120-bit | 5120-bit | 5120-bit |
| Memory Bandwidth | 3 TB/s | 2 TB/s | 2 TB/s |
| TDP | 700W | 350W | 400W |
| FP64 | 30 TFLOPS | 24 TFLOPS | 9.75 TFLOPS |
| FP32 | 60 TFLOPS | 48 TFLOPS | 19.5 TFLOPS |
| FP64 Tensor Core | 60 TFLOPS | 48 TFLOPS | 19.5 TFLOPS |
| TP32 Tensor Core | 500 TFLOPS | 400 TFLOPS | 156 TFLOPS |
| FP16 Tensor Core | 1,000 TFLOPS | 800 TFLOPS | 312 TFLOPS |
| FP8 Tensor Core | 2,000 TFLOPS | 1,600 TFLOPS | N/A |
A tremendous performance leap
With Hopper, Nvidia has delivered an incredible performance boost to its HPC customers. Even with its TDP restricted PCIe version, Hopper delivers users a huge performance boost over Nvidia's Ampere-based A100 silicon. Simply put, Hopper represents one of Nvidia's largest generational leaps in performance to date.
Nvidia expects to start shipping Hopper H100 accelerators in Q3 2022.
You can join the discussion on Nvidia's Hopper H100 accelerators on the OC3D Forums .