STAC Study - A Proof-of-Concept for LSTM Inference on Market Data

Financial firms on the STAC Benchmark Council requested benchmarks for realtime inference on market data. Using their input, STAC designed this study to initiate the development of an inference benchmark and elicit feedback from the broader STAC Benchmark Council. While STAC developed the proof-of-concept (POC) benchmark specifications used in the study, they are not official benchmarks from the STAC Benchmark Council. The Council subsequently evolved the POC into formalized benchmark specifications. If you would like to be involved in the ongoing evolution of these and other AI/ML benchmarks, please join the STAC AI Working Group.

We ran the benchmarks on multiple hardware and software configurations to ascertain how useful the POC benchmarks were for comparing architectural solutions. We used eight different Dell PowerEdge C6525 dual-socket servers with 256 GB of memory and varying AMD CPU models and configurations.

For a given hardware platform, the objective was to measure the upper-bound performance of inference in three configurations corresponding to different user preferences for latency/throughput tradeoffs.

Though no vendors had a hand in optimizing the systems' performance, one vendor did help make the project happen: Dell provided access to the Dell Technologies HPC and AI Innovation Lab, which includes clusters of computers with many hardware variations from their technology partners. We are grateful for their help.

The STAC Study is available to subscribers of the Analytics STAC Track. The implementation code in the STAC source code repository and the dataset are also available. Please contact us for access.