STAC-Reports: OpenShift vs bare metal on a modern low-latency stack

OpenShift-based system showed same (record) mean latencies as bare metal

13 June 2018

STAC has released two sets of audited STAC-N1™ results from a pair of overclocked Supermicro servers using a Solarflare network stack and RHEL 7.5:

• The first set of results was from the system run as bare metal. Those results are available here.
• The second set of results was from the system using containers and orchestrated by Red Hat OpenShift 3.10 (pre-release). Its results are available here.

STAC-N1 measures the performance of a host network stack (server, operating environment, drivers, host adapter) using a market data style workload.

This benchmark was performed on a pair of Supermicro SYS-1029UX-LL1-S16 servers, each with 2 x 8-core Intel Xeon Scalable 6144 (Gold) processors overclocked @ 4.18 GHz. In the servers were Solarflare X2522 Adapters and RHEL 7.5.

The software stack also included Red Hat OpenShift Container Platform 3.10 (pre-release), along with a newly created Solarflare Kubernetes Device Plugin and pre-start hook mechanism to ease use of OpenOnload accelerated containers. (NOTE: Since the OpenShift configuration included components that are not currently released, it's possible that the performance of the solution tested here will differ once they are released.) The system could switch back and forth between bare metal and containers through a simple configuration script.

Notable results included:

• Compared to all prior publicly released STAC-N1 results, the bare metal system demonstrated the lowest mean latency at both the base rate of 100K messages per second and the highest rate tested (1 million mps). In addition, max latency at 100K mps was the lowest of any system using sockets.
• Mean and 99th percentile latency of the system using Red Hat OpenShift were the same as on bare metal at both 100K mps and 1 million mps.

The reports analyze latencies and other metrics in detail. Premium subscribers have access to a CSV file with results for several message sizes beyond the standard size used for the report, a detailed STAC Configuration Disclosure, a complete sosreport for the system, a detailed analysis of Spectre/Meltdown mitigations, and the OpenShift configuration files.

If your firm does not have access to these materials, please take a minute to learn about subscription options.