Reside Intel 4th Gen Xeon Benchmarks: Sapphire Rapids Accelerators Revealed

This afternoon at Intel Innovation 2022 in San Jose, Intel supplied a sneak peek of the capabilities of the on-board accelerators of its forthcoming 4th Technology Xeon Processor household, generally known as Sapphire Rapids. In actual fact, we have been handled to a hands-on “Intel 4th Gen Xeon Accelerator Expertise” at this time, to look at Sapphire Rapids in motion versus AMD’s EPYC 7763 64-core Milan processor. We have been invited into an information middle server room setup, with the pleasant sound of rack servers screaming, to witness firsthand stay benchmark runs of particular accelerated workloads.

The workloads to be employed have been comprised of widespread cloud information middle duties like information compression/decompression, IPSec encryption throughput, AI picture classification, safe net server SSL handshaking, and database question efficiency. Nevertheless, earlier than we dig into the main points, let’s shortly recap what we’ve lined earlier than about Intel’s 4th Gen Xeons.

Intel 4th Gen Xeon Scalable Accelerators Detailed

With Sapphire Rapids, it’s not simply concerning the chips’ enhanced Intel 7 process-built Golden Cove CPU cores. This new Intel 4th Gen Xeon Scalable server CPU microarchitecture has devoted {hardware} accelerators on board that may present important efficiency uplifts in an array of widespread information middle workloads and functions. 

Extra particularly, in silicon, Sapphire Rapids brings help for Intel AMX (Superior Matrix Extensions) for machine studying and AI, a Dynamic Load Balancing (DLB) accelerator for safety and gateway load balancing, a Knowledge Streaming Accelerator (DSA) for networking and storage offload, Intel’s In-Reminiscence Analytics Accelerator (IAA) for database processing and throughput acceleration, AVX-512 help once more for analytics and deep studying, and Intel Fast Help Expertise (QAT) for accelerating compression, NGINX, OpenSSL, and IPSec (safety) workloads. 4 of these six accelerators usually are not obtainable on third Gen Xeon Scalable platforms.

Whereas Intel wasn’t demonstrating common function compute workloads on these pre-production 4th Gen Xeon chips and servers in our demo, the corporate was particularly showcasing the efficiency good points that using these particular on-board engines can provide to those widespread information middle workloads and duties. So, you may say this was a really managed head-to-head versus AMD EPYC, however the accelerators in motion on the workloads employed are additionally key to advancing information middle server efficiency, performance-per-watt metrics and TCO.

Intel 4th Gen Xeon Scalable Sapphire Rapids 2P Server

All check outcomes under have been carried out in September 2022 on programs Intel configured, and with that backdrop set and people config particulars, let’s get to some benchmark information.

Intel 4th Gen Xeon Scalable Accelerator Benchmarked

Using a pretrained deep studying AI mannequin for picture classification, Intel demonstrated efficiency good points with simply AVX-512 and Intel VNNI (Vector Neural Community Directions) AVX-2 acceleration, after which once more with AVX-512 and Intel AMX (Superior Matrix Extensions) performing a Tile Matrix Multiply to speed up issues additional. Listed here are the outcomes…

ResNet50v1.5 Tensorflow AI Picture Classification Efficiency With Intel AMX

As you’ll be able to see, there’s a dramatic speed-up in variety of photos processed per second, and an enormous discount in latency with Intel VNNI employed alone with INT8 precision. Nevertheless, kick-in the 4th Gen Xeon’s AMX matrix multiply array and the check confirmed an approximate 6X carry in efficiency. We watched these checks working stay and may the truth is confirm these outcomes, at the very least for the servers within the rack that have been being examined in entrance of us.

QATzip Degree 1 Compression Acceleration Benchmarks

QATzip is a person area library that may leverage the Intel QuickAssist Expertise to speed up file compression and decompression providers, by offloading the precise compression and decompression requests to the devoted accelerator in Intel’s 4th Gen Xeon Scalable processors.

At first look, the accelerated QATzip compression workload seems to be solely marginally sooner than the AMD EPYC configuration using 120 cores with ISA-L (Clever Storage Acceleration Library). However what the numbers present is that when using Intel QuickAssist Expertise (QAT), the workload is usually offloaded from the CPU cores. Solely 4 cores and the QAT accelerator are utilized within the quickest 4th Gen Xeon Scalable configuration, releasing up 116 of the CPU cores for different duties and VM availability.

ClickHouse Large Knowledge Analytics Benchmarks On Sapphire Rapids

ClickHouse is an Open-source column-oriented database administration system for on-line analytical processing. It could possibly run on Naked metallic, within the Cloud (AWS, AliCloud, Azure, and so on.), or containerized in Kubernetes. It’s linearly scalable and will be scaled as much as retailer and course of huge quantities of knowledge. These outcomes have been generated utilizing a Star Schema Benchmark, centered on Question This fall.1, which has the very best CPU utilization…

There are a number of outcomes represented right here, displaying the 60-core 4th Gen Xeon Scalable system using IAA outperforming the 64-core EPYC configuration by way of queries per second and compression fee. The 4th Gen Xeon Scalable system additionally utilized much less reminiscence and fewer reminiscence bandwidth in the course of the benchmark, releasing up these resourced for different duties.

Intel 4th Gen Xeon IAA Accelerating RocksDB

RocksDB is an embedded persistent key-value retailer used as storage engine in lots of in style databases (MySQL, MariaDB, MongoDB, Redis, and so on.). It is utilized in Fb’s internet hosting setting, for instance, as effectively. For these benchmarks, RocksDB was modified to help compressors as plugins. Though not publicly obtainable simply but, the code might be upstreamed to the RocksDB venture…

On this benchmark, the 60-core 4th Gen Xeon Scalable configuration using IAA almost doubles the efficiency of the 64-core EPYC system, whereas concurrently providing a lot decrease latency.

SPDK NVMe TCP Storage Efficiency With Knowledge Safety

This subsequent benchmark is illustrating Error Safety of NVMe TCP information at 200Gbps. For this check, the FIO submits I/O requests to a SPDK (Storage Efficiency Growth Equipment) NVMe/TCP goal. The goal reads information from NVMe SSDs and makes use of DSA or ISA-L for on the Intel processor and ISA-L on the AMD processor to calculate CRC32C information digest. Then the goal sends I/O and CRC32C information digest to FIO…

Whether or not using a single core with 128K sequential reads (QD64) or two cores with 16K random reads (QD256), the Intel 4th Gen Xeon server provides considerably greater throughput, at a lot decrease latencies than the EPYC server.

Intel 4th Gen Xeon Scalable IPSec Encryption Benchmarks

The DPDK IPsec-GW (safety gateway) benchmark measures how a lot site visitors the server can course of per second utilizing the IPsec protocol. Encryption is dealt with by software program utilizing Intel Multi-Buffer Crypto for IPsec library or offloaded to the Intel QAT accelerator. The Intel IPsec library implements optimized encryption/decryption operations utilizing AES or VAES directions, however observe VAES directions can’t be utilized on the AMD system as a result of AVX-512 isn’t obtainable on third Gen EPYC processors…

At first look, the bars on this chart do not present massive disparities between the totally different configurations. Nevertheless, the Xeon system is ready to obtain considerably greater efficiency, whereas using far fewer cores.

Intel Xeon NGINX Key Handshake OpenSSL Crypto Acceleration Assessments

This NGINX TLS Key Handshake benchmark measures encrypted Net Server connections-per-second. No packets are requested by the purchasers, so solely a TLS handshake is full. The benchmark stresses the server’s compute, reminiscence, and IO assets…

As soon as once more the testing reveals the 4th Gen Xeon Scalable system providing comparable efficiency to the EPYC system, however when using Intel QAT (SW or HW), the Xeon configuration is ready to acquire that efficiency whereas taxing far fewer CPU cores.

All of those benchmarks have been devised by Intel to obviously reveal the potential efficiency and effectivity advantages of the assorted accelerators obtainable in its upcoming 4th Gen Xeon Scalable processors, in any other case generally known as Sapphire Rapids. The checks have been run beneath managed circumstances and a lot of the particulars relating to Intel’s 4th Gen Xeon Scalable processors being employed within the check servers weren’t divulged. With all of that in thoughts, we clearly need to mood our evaluation of those numbers, however assuming every little thing proven holds up in the true world once we do get to independently confirm these outcomes, they definitely bode very effectively for Intel.

AMD will retain a CPU core density per socket benefit with its EPYC processors, but when these devoted accelerators in 4th Gen Xeon Scalable processors negate that benefit and unlock Intel CPU core assets for different duties whereas accelerating these widespread workloads, this new Intel Knowledge Heart Group product providing may have main implications for tomorrow’s information facilities.