Intel Hits Back At AMD’s Data Center AI Performance Claims: Says 5th Gen Xeon Faster Than AMD EPYC Turin Using Proper Optimizations

Intel has followed up on AMD's Data Center AI performance claims for its 5th Gen EPYC Turin CPUs, saying 5th Gen Xeon is faster with the right optimizations.

AMD Showed Its 5th Gen EPYC Turin CPUs With Zen 5 Cores Beating 5th Gen Xeon Chips In AI Benchmarks But Intel Says That 5th Gen Leads Against EPYC When Right Optimizations Are Used

At Computex 2024, AMD officially announced its 5th Gen EPYC CPU family codenamed Turin which utilizes the latest Zen 5 core architecture. The company put out some big numbers against the Intel 5th Gen Xeon family codenamed Emerald Rapids, specifically in AI throughput workloads but Intel has now clarified that those benchmarks were conducted without the proper optimizations for its Xeon family, and with those implemented, even 5th Gen Xeons should easily surpass 5th Gen EPYC Turin in AI performance.

Related Story TSMC’s 3nm Supply Runs Short Amid Huge AI Demand: NVIDIA, Apple, AMD, Qualcomm Consider Raising Chip Prices

The benchmark that was specifically highlighted by Intel is the Llama2-7B Chatbot which is based around the INT4 inference throughput and was conducted at a 50ms latency. AMD showed that its 5th Gen EPYC CPUs in a 2S (dual-socket) configuration with 128 cores each offer up to 671 tokens/s of performance while Intel's 5th Gen Xeon Platinum 8592+ chips with 64 cores each running in the same dual-socket configuration offered just 125 tokens/s output. That was a huge 5.4x gain for the AMD EPYC Turin CPUs.

Intel states that the benchmarks were conducted without the right software suite for the 5th Gen Xeon SKUs and AMD didn't put out any details regarding the Intel configuration in its footnotes.

The blue team has now conducted its performance benchmarks in the same AI workloads & it looks like the results are radically different.

Using the Intel Extension for PyTorch (P99 Latency), the 5th Gen Emerald Rapids Xeon CPUs can output 5.4x better performance than what AMD showcased. The 686 Tokens/s output exceeds the