STAC Report: kdb+ with Pavilion Data Hyperparallel Flash Array under STAC-M3 (tick analytics)

Solution sets 4 records in baseline tests.

14 October 2019

STAC recently performed STAC-M3 Benchmarks on a stack involving the kdb+3.6 database management system with the Pavilion Data Hyperparallel Flash Array. The STAC Report is available here.

STAC-M3 is the set of industry standard enterprise tick-analytics benchmarks for database software/hardware stacks that manage large time series of market data ("tick data"). In this project, we ran the baseline benchmark suite (code named Antuco).

In the stack under test, Kx's kdb+ 3.6 database resided on a Pavilion Data Hyperparallel Flash Array (HFA) 2.2.1 containing a mix of SSDs that provided a total of ~100TB of usable space with RAID-6. The kdb+ DBMS was distributed across 6 dual-socket servers that accessed the HFA via RoCE v2 over 100G Ethernet. Pavilion Data chose to highlight that this solution:

• Outperformed all publicly disclosed results in 4 of 17 mean-response time benchmarks:
  • STAC-M3.β1.100T.STATS-UI.TIME
  • STAC-M3.β1.50T.STATS-UI.TIME
  • STAC-M3.β1.10T.STATS-UI.TIME
  • STAC-M3.β1.10T.STATS-AGG.TIME
• Was faster in 8 of 17 mean-response time benchmarks than all other solutions using kdb+ and flash arrays, including the benchmarks above plus:
  • STAC-M3.β1.100T.VWAB-12D-NO.TIME
  • STAC-M3.β1.10T.VOLCURV.TIME
  • STAC-M3.β1.1T.MOHIBID.TIME
  • STAC-M3.β1.1T.STATS-UI.TIME
• Was faster in 10 of 17 mean-response time benchmarks versus a previous record-setting kdb+ solution involving a competitive flash array (SUT ID KDB170421), including:
  • 2x the speed for STAC-M3.β1.10T.STATS-AGG.TIME
  • Over 1.7x the speed for both STAC-M3.β1.10T.MKTSNAP.TIME and STAC-M3.β1.10T.STATS-UI.TIME
  • Over 1.5x the speed for both STAC-M3.β1.100T.STATS-UI.TIME and STAC-M3.β1.50T.STATS-UI.TIME.

Details are in the STAC Report at the link above. Premium subscribers have access to the code used in this project and the micro-detailed configuration information for the solution. (To learn about subscription options, please contact us.)