Intel Gaudi 2 Accelerators Showcase Competitive Performance Per Dollar Against NVIDIA H100 In MLPerf 4.0 GenAI Benchmarks

Intel has just released its latest MLPerf v4.0 performance figures covering the Gaudi 2 Accelerators & 5th Gen Xeon "Emerald Rapids" CPUs, with the former showcasing strong performance per dollar values against NVIDIA's H100 GPU.

Intel Gaudi 2 AI Accelerator Is The Only "Benchmarked Alternative" To NVIDIA's H100 GPUs For GenAI, Strong Perf/$ In MLPerf v4.0 Results

Intel has been fine-tuning the performance of its Gaudi accelerator lineup in AI workloads using its OneAPI framework for some time now. The result of this ongoing software work was showcased in the latest MLPerf v4.0 performance figures which showcase the GenAI capabilities in workloads like Llama-70B and Stable Diffusion XL where Intel's solutions offer competitive performance against its rivaling chips. More recently, the company showcased how Gaudi 2 accelerators were faster versus NVIDIA's solutions in the latest GenAI workloads such as Stable Diffusion & Llama 2 LLMs. More on that here.

Related Story Intel Lunar Lake-MX CPU Pictured On Reference Platform: 8 CPU Cores, 8 Arc GPU Cores, On-Package LPDDR5x Memory

For comparisons, Intel used an x8 Gaudi 2 accelerator configuration against x8 NVIDIA H100 GPUs for FP8 and INT8 performance benchmarking. In relative performance, the NVIDIA H100 without a doubt sits much ahead of the Intel Gaudi 2 accelerators, offering up to 3.35x uplifts in server & up to 2.76x uplifts in offline generation. But where the game completely shifts in Intel's favor is the perf/$ where the Gaudi 2 accelerators become a very competitively positioned product and what Intel terms Gaudi 2 as the only "Benchmarked Alternative" to NVIDIA's H100 for GenAI workloads.

So in terms of performance per dollar, the Intel Gaudi 2 AI accelerator offers 33% better value versus the NVIDIA H100 solution with the NVIDIA H100 only outpacing Gaudi 2 in Llama-70B (server). Intel has also recently partnered with Qualcomm and Google to tackle NVIDIA's CUDA dominance in AI through oneAPI which can lead