NVIDIA Blackwell Debuts at MLPerf & Shatters AI Performance Records, Hopper’s Leadership Continues With H100 & H200 Outperforming AMD MI300X

NVIDIA's Blackwell AI chips secure record performance at MLPerf while Hopper H100 & H200 chips continue to get even stronger, exceeding the MI300X.

NVIDIA's Dominance In MLPerf Inference & AI Benchmarks Solidified Further With Blackwell AI Chips, Hopper Show Even Stronger Performance Uplifts Thanks To Continued Optimizations To The CUDA Stack

NVIDIA's Blackwell AI chips have finally made their record debut in MLPerf v4.1, securing record performance numbers across all benchmarks. Coming to data centers later this year, the NVIDIA Blackwell AI chips are poised as the strongest AI solution on the market, with up to 4x increase in generational performance.

Related Story SMC Offers “Immersion Cooling” In NVIDIA’s Blackwell AI Servers, Scaling Down Power Consumption By 50%

Today, NVIDIA announced that it has achieved the highest performance in MLPerf Inference v4.1 across all AI benchmarks which include:

• Llama 2 70B (Dense LLM)
• Mixtral 8x7B MoE (Sparse Mixture of Experts LLM)
• Stable Diffusion (Text-to-Image)
• DLRMv2 (Recommendation)
• BERT (NLP)
• RetinaNet (Object Detection)
• GPT-J 6B (Dense LLM)
• 3D U-Net (Medical Image Segmentation)
• ResNet-50 v1.5 (Image Classification)

In Llama 2 70B, NVIDIA's Blackwell AI solutions offer a massive increase over the Hopper H100 chips. In server workloads, a single Blackwell GPU offers a 4x increase (10,756 Tokens/second) while in Offline scenarios, the single Blackwell GPU offers a 3.7x increase in performance with 11,264 Tokens/second. NVIDIA also offered the first publicly measured performance using FP4 running on Blackwell GPUs.

While Blackwell is the beast it was promised to be, NVIDIA's Hopper continues to get even stronger with more optimizations landing through the CUDA stack. The H200 and H100 chips offer leading performance across every test compared to the competition and also in the latest benchmarks such as the 56-billion parameter "Mixtral 8x7B" LLM.

The NVIDIA HGX H200 with 8 Hopper H200 GPUs and NVSwitch offers strong