A STAC Test Harness™ is software that enables the execution of STAC Benchmarks. A STAC Pack™ is software that enables a given STAC Test Harness to conduct tests of a specific product (depending on the benchmark, this might be a particular database, a particular processor, etc.). Each STAC Test Harness embodies the industry’s best practices in four areas: workload generation, scenario configuration, measurement, and analysis & reporting.
The code that makes up STAC Test Harnesses and STAC Packs is contributed by STAC and members of the STAC Benchmark Council. Other members of the Council can use this software to run STAC Benchmarks in the privacy of their own labs. This provides an insightful, standardized picture of performance that is capable of exact comparison with published test results. These tools can also be used to perform customized tests rather than the standard benchmarks (e.g., testing boundary conditions, exceptions, sensitivity, etc.).
For more background on STAC Test Harnesses and STAC Packs, please see the FAQ.
Artificial Intelligence
STAC-AI (Inferencing) STAC-AI™ Test Harness
Machine Learning
Inference (STAC-ML) STAC-ML Markets (Inference) Test Harness STAC-ML Markets (Inference) Reference Implementation (ONNX & TensorFlow) STAC-ML Pack for CUDA and cuDNN STAC-ML Pack for Myrtle.ai VOLLO STAC-ML Pack for GroqWare
Fast Workloads
Time sync (STAC-TS) STAC-TS Test Harness
Tick-to-trade (STAC-T1) STAC-T1 Test Harness
Network I/O (STAC-N1, -T0) STAC-N Test Harness STAC-N Binding for Sockets STAC-N Binding for RDMA STAC-N Binding for RDS STAC-N1 Pack for Lightfleet STAC-N2 Test Harness STAC-T0 Test Harness
Data distribution (STAC-M2) STAC-M2 Test Harness STAC-M2 Advanced Test Harness STAC-M2 Pack for Thomson Reuters UPA STAC-M2 Pack for Thompson Reuters RFA
Big Workloads
Risk computation (STAC-A2) STAC-A2 Test Harness STAC-A2 Pack for Julia STAC-A2 Pack for Intel STAC-A2 Pack for CUDA STAC-A2 Pack for Mathematica STAC-A2 Pack for C – Baseline STAC-A2 Pack for IBM Power STAC-A2 Pack for NEC Vector Engine STAC-A2 Pack for AMD Xilinx Vitis
Backtesting (STAC-A3) STAC-A3 Test Harness STAC-A3 Pack for Spark STAC-A3 Pack for Python with RAPIDS STAC-A3 Pack for Intel
Tick analytics (STAC-M3) STAC-M3 Test Harness STAC-M3 Pack for kdb+ STAC-M3 Pack for shakti STAC-M3 Pack for eXtremeDB Financial Edition