STAC Report: kdb+ on WekaFS and HPE under STAC-M3 (tick analytics)

Solution sets total of 17 records in scale and baseline benchmarks

29 May 2020

STAC recently performed STAC-M3 Benchmarks on a stack involving kdb+, the WekaIO WekaFS parallel filesystem, and a server cluster from HPE. The results are available here.

STAC-M3 is the set of industry standard enterprise tick-analytics benchmarks for database software/hardware stacks that manage large time you series of market data ("tick data"). In this project, we ran both the baseline benchmark suite (code named Antuco) and the optional scaling suite (code named Kanaga).

The stack under test was Kx's kdb+ 3.6 database system distributed across 14 HPE Proliant XL170r Gen10 servers, with data stored in a cluster of 18 HPE Proliant XL170r Gen10 servers with a total of 251TiB of SSD capacity, all accessed via WekaIO WekaFS 3.6.2 software. WekaIO chose to highlight that this solution:

• Outperformed all publicly disclosed results in 11 of 24 Kanaga mean-response time (MRT) benchmarks:
  • 100-user 12-day VWAB: all 5 benchmark years (STAC-M3.β1.100T.YR[n].VWAB-12D-HO.TIME)
  • 50-user 12-day VWAB: benchmark years 4 and 5 (STAC-M3.β1.50T.YR4VWAB-12D-HO.TIME and STAC-M3.β1.50T.YR5VWAB-12D-HO.TIME).
  • Multi-year high bid: all 4 multi-year spans (STAC-M3.β1.1T.[n]YRHIBID.TIME)

• Outperformed all publicly disclosed results in all Kanaga throughput benchmarks (STAC-M3.β1.1T.*.BPS)

• Versus a kdb+ solution involving an all-flash NAS and 4 database nodes (SUT ID KDB190430):
• was faster in all 24 Kanaga MRT benchmarks; and
• was faster in 15 of 17 MRT Antuco benchmarks, including:
• 8.8x speedup in 100-user interval stats (STAC-M3.β1.100T.STATS-UI.TIME)
• 4.5x speedup in 10-user aggregate stats (STAC-M3.β1.10T.STATS-AGG.TIME)

• Versus a kdb+ solution involving a single server with direct-attached Intel Optane and 3D-NAND Flash SSD (SUT ID KDB181009):
• was faster in 19 of 24 Kanaga MRT benchmarks, including:
• 20.3x speedup in STAC-M3.β1.100T.YR2VWAB-12D-HO.TIME; and
• was faster in 4 of 17 MRT Antuco benchmarks.

• Versus a kdb+ solution involving a Fibre Channel-connected flash array and 4 database nodes (SUT ID KDB170421):
• was faster in 12 of 24 Kanaga MRT benchmarks and 3 of 17 Antuco MRT benchmarks.

Details are in the STAC Report at the link above. Premium subscribers have access to the code used in this project and the micro-detailed configuration information for the solution.(To learn about subscription options, please contact us.)