Dissecting Intel's EPYC Benchmarks: Performance Through the Lens of Competitive Analysis
by Johan De Gelas & Ian Cutress on November 28, 2017 9:00 AM EST- Posted in
- CPUs
- AMD
- Intel
- Xeon
- Skylake-SP
- Xeon Platinum
- EPYC
- EPYC 7601
Although the AMD EPYC is definitely a worthy contender in the server space, AMD's technical marketing of the new CPU has been surprisingly absent, as the company not published any real server benchmarks. The only benchmarks published were SPEC CPU and Stream, with AMD preferring for its partners and third parties to promote performance. And, as our long-time readers know, while the SPEC CPU benchmarks have their merits and many people value them, they are a very poor proxy of most server workloads.
In every launch, we expect companies to offer an element of competitive analysis, often to show how their platform is good or better than the rest. At the launch of Intel's latest Xeon-SP platform, analysis to EPYC was limited to a high-level, as the systems were not as freely available as expected. AMD was able to do so on Broadwell-E at the time of the EPYC announcement because it was out and available - Intel wasn't able to do it on EPYC because AMD were several months away from moving it from a cloud-only ramp up program. This is partly the effect of AMD's server market implementation and announcement roadmap, although it didn't stop Intel from hypothesising about the performance deficits in ways that caught the attention of a number of online media.
Throughout all of this, AMD could not resist but to continue to tell the world that the "EPYC SoC Sets World Records on SPEC CPU Benchmarks". In the highly profitable field that is server hardware, this could not be left unanswered by Intel, who responded that the Intel Xeon Scalable has great "momentum" with no less than 110 performance records to date.
Jumping to the present time, in order to to prove Xeon-SP dominance over the competition, Intel's data center engineering group has been able to obtain a few EPYC systems and has started benchmarking. This benchmarking, along with justifications of third-party verification, was distributed to the small set of Xeon-SP launch reviewers as a guide, to follow up on that high-level discussion some time ago. The Intel benchmarking document we received had a good amount of detail however, and the conference call we had relating to it was filled with some good technical tidbits.
Our own benchmarks showed that the EPYC was a very attractive alternative in some workloads (Java applications), while the superior mesh architecture makes Intel's Xeon the best choice in other (Databases for example).
A Side Note About SPEC
A number of these records were achieved through SPEC. As mentioned above, while SPEC is a handy tool for comparing the absolute best tweaked peak performance of the hardware underneath, or if the system wants to be analysed close to the metal because of how well known the code base is, but this has trouble transferring exactly to the real world. A lot of time the software within a system will only vaguely know what system it is being run on, especially if that system is virtualised. Sending AVX-512 commands down the pipe is one thing, but SPEC compilation can be tweaked to make sure that cache locality is maintained whereas in the real-world, that might not be possible. SPEC says a lot about the system, but ultimately most buyers of these high-end systems are probing real-world workloads on development kits to see what their performance (and subsequent scale-out performance) might be.
For the purposes of this discussion, we have glossed over Intel's reported (and verified over at SPEC.org) results.
Pricing Up A System For Comparison
Professionals and the enterprise market will mention, and quite rightly, that Intel has been charging some heavy premiums with the latest generation, with some analysts mentioning a multiple jump up in pricing even for large customers, making it clear that the Xeon enterprise CPU line is their bread and butter. Although Intel's top-end Xeon Platinum 8180 should give the latest EPYC CPU a fit of trouble thanks to its 28 Skylake-SP cores running at 2.5 to 3.8 GHz, the massive price tag ($10009 for the standard version, $13011 for the high-memory model) made sure that Intel's benchmarking team had no other choice than also throwing in a much more modest Xeon Platinum 8160 (24 cores at 2.1 - 3.7 GHz, $4702k) as well as the Xeon Gold 6148 (20 cores at 2.4-3.7 GHz, $3072).
SKUS Tested | |||||
Intel Xeon Platinum 8180 |
Intel Xeon Platinum 8160 |
Intel Xeon Gold 6148 |
AMD EPYC 7601 |
||
Release Date | Early Q3, 2017 | Late Q2, 2017* | |||
Microarchitecture | Skylake-SP with AVX-512 | Zen | |||
Process Node | Intel 14nm (14+) | GloFo 14nm | |||
Cores / Threads | 28 / 56 | 24 / 48 | 20 / 40 | 32 / 64 | |
Base Frequency | 2.5 GHz | 2.1 GHz | 2.4 GHz | 2.2 GHz | |
Turbo | 3.8 GHz | 3.7 GHz | 3.7 GHz | 3.2 GHz | |
L2 Cache | 28 MB | 24 MB | 20 MB | 16 MB | |
L3 Cache | 38.5 MB | 33.0 MB | 27.5 MB | 64 MB | |
TDP | 205 W | 150 W | 150 W | 180 W | |
PCIe Lanes | 48 (Technically 64 w/ Omni-Path Versions) | 128 | |||
DRAM | 6-channel DDR4 | 8ch DDR4 | |||
Max Memory | 768 GB | 2048 GB | |||
Price | $10009 | $4702 | $3072 | $4200 |
As a result of this pricing, one of the major humps for Intel in any comparison will be performance per dollar. In order to demonstrate that systems can be equivalent, Intel offered up this comparison from a single retailer. Ideally Intel should have offered multiple configurations options for this comparison, given that a single retailer can intend for different margins on different sets of products (or have different levels of partnership/ecosystem with the manufacturers).
Even then, price parity could only be reached by giving the Intel system less DRAM. Luckily this was the best way to configure the Intel based system anyway. We can only guess how much the benchmarking engineers swore at the people who set the price tags: "this could have been so much easier...". All joking apart, the document we received had a good amount of detail, and similar to how we looked into AMD's benchmarking numbers at their launch, we investigated Intel's newest benchmark numbers as well.
105 Comments
View All Comments
Ashari - Tuesday, November 28, 2017 - link
LOL, "GloFo 16nm"... tsts, one would think people like Johan De Gelas and Ian Cutress would know which node is GloFo and which one is TSMCIan Cutress - Tuesday, November 28, 2017 - link
That's my brain fart. I've been writing about other things recently. Edited.peevee - Tuesday, November 28, 2017 - link
"The benchmarking scenario also has a big question mark, as in the footnotes to the slides Intel achieved this victory by placing 58 VMs on the Xeon 8160 setup versus 42 VMs on the EPYC 7601 setup."Given how well AMDs SMT scales, a real client can put up to 128 single-CPU VMs on the EPIC 7601, and 58 VMs on Xeon 8160 would be tramped ridiculously.
Here Intel just had to rely on the shenanigans so obvious it is just fraud.
LordOfTheBoired - Tuesday, November 28, 2017 - link
Yeah, that really stuck out for me too. "We outperform AMD when running a different benchmark!"And to be frank, it casts a pall over Intel's entire PR release since it IS blatantly not how benchmarks work.
Andresen - Tuesday, November 28, 2017 - link
Many HPC task are memory bandwidth limited, and then AVX-512 is of little help. In Spec.org CFP2006 none of the recent results are using AVX-512 but instead rely on AVX2. The few tests posted using AVX-512 come out worse than the tests on similar systems using AVX2. For memory bandwidth limited tasks the EPYC has an advantage with its 8 memory channels compared to Intels 6 channels. For both architectures, a high end processor is not needed for bandwidth limited task, since the don't offer more memory channels.Johan Steyn - Monday, December 18, 2017 - link
AVX also heats up the CPU a lot and it has to throttle down. With AVX, Intel cannot run high clock speesds.ddriver - Tuesday, November 28, 2017 - link
Just when you think AT cannot possibly sink any lower, they now directly publish publish intel benchmarks of a competing product.Coldfriction - Tuesday, November 28, 2017 - link
I myself was confused and dissapointed reading the summary where agreement with Intel seems to be presented by the authors. Using prases like "there is no denying that the Intel Xeon is a 'safer bet' for VMware virtualization" without testing it pushes AT into the realm of paid for shills. Independent reviews wouldn't trust anyone's marketing and even if they were to publish an article on benchmarks from a competitor, they would fill the thing with hefty amounts of skepticism until they could test it themselves. What Intel presents could very realistically be true (personally, I don't doubt that their benchmarks are within the ballpark of being legit), but I want my independent review sites to have as little bias as possible and that means objectively testing the hardware and ignoring the marketing.wumpus - Wednesday, November 29, 2017 - link
These type of servers are rarely bought by customers for personal use. Instead, they are bought for a 'real job' where CYA decisions outweigh any performance benefits (to a degree, the end product has to work). If something really goes wrong, you can always expect to get the blame for buying the "off brand" instead of following the sheep, regardless of what really caused the failure (typically with highly annoyed management who can't tell *anything* about the server than it is the "off brand").If this isn't a consideration you have a "great job". Expect the owner to sell at some point or expand to the point it is controlled by MBAs and downgrade everybody's job to a "real job'. Sorry to say, but at least in the USA that is life.
Johan Steyn - Monday, December 18, 2017 - link
People sometimes really surprise me. What support doe you want from AMD? Yes if there is a booboo like Intel has (present tense) with its security flaw, you need support from them. I have sold numerous systems and servers in my life and never did I go to AMD or Intel to ask for support. It either the OEM, component supplier or component manufacturer (like motherboards etc) who you go to for support.If the CPU works as it should, you do not need support. CPU's were in my experience the one component that rarely if ever dies on you. So if you trust Tyan to make good products, which they do, they are the ones to give you support, not AMD. AMD has to help with Bioses etc. with which they are very good.
So please stop with this support issue and safer bet. If the system runs unstable because of hardware issues, sure they have to sort it out, but till now, none has been reported.
What has Intel done about the bug recently found? Did they come to you to fix it and support you? Nope, you have to fix it yourself, that is if the motherboard manufacturer has a bios update. So, for me it looks like AMD might just be the safer bet after all...