A STAC Test Harness™ is software that enables the execution of STAC Benchmarks. A STAC Pack™ is software that enables a given STAC Test Harness to conduct tests of a specific product (depending on the benchmark, this might be a particular database, a particular processor, etc.). Each STAC Test Harness embodies the industry’s best practices in four areas: workload generation, scenario configuration, measurement, and analysis & reporting.
The code that makes up STAC Test Harnesses and STAC Packs is contributed by STAC and members of the STAC Benchmark Council. Other members of the Council can use this software to run STAC Benchmarks in the privacy of their own labs. This provides an insightful, standardized picture of performance that is capable of exact comparison with published test results. These tools can also be used to perform customized tests rather than the standard benchmarks (e.g., testing boundary conditions, exceptions, sensitivity, etc.).
For more background on STAC Test Harnesses and STAC Packs, please see the FAQ.
Inference (STAC-ML) STAC-ML Markets (Inference) Test Harness STAC-ML Markets (Inference) Reference Implementation (ONNX & TensorFlow) STAC-ML Pack for CUDA and cuDNN STAC-ML Pack for Myrtle.ai VOLLO STAC-ML Pack for GroqWare
Time sync (STAC-TS) STAC-TS Test Harness
Tick-to-trade (STAC-T1) STAC-T1 Test Harness
Network I/O (STAC-N1, -T0) STAC-N Test Harness STAC-N Binding for Sockets STAC-N Binding for RDMA STAC-N Binding for RDS STAC-N1 Pack for Lightfleet STAC-T0 Test Harness
Feed handlers (STAC-M1) STAC-M1 Test Harness STAC-M1.OPRA
Data distribution (STAC-M2) STAC-M2 Test Harness STAC-M2 Advanced Test Harness STAC-M2 Pack for Thomson Reuters UPA STAC-M2 Pack for Thompson Reuters RFA
Risk computation (STAC-A2) STAC-A2 Test Harness STAC-A2 Pack for Julia STAC-A2 Pack for Intel Composer XE STAC-A2 Pack for CUDA STAC-A2 Pack for Mathematica STAC-A2 Pack for C – Baseline STAC-A2 Pack for IBM Power
Backtesting (STAC-A3) STAC-A3 Test Harness STAC-A3 Pack for Spark STAC-A3 Pack for Python with RAPIDS STAC-A3 Pack for Intel
Tick analytics (STAC-M3) STAC-M3 Test Harness STAC-M3 Pack for kdb+ STAC-M3 Pack for shakti STAC-M3 Pack for eXtremeDB Financial Edition