• Contact
  • Register
    (If you're a human, don't change the following field)
    Your first name.
    (If you're a human, don't change the following field)
    Your first name.

    Enter your email above, then click "Sign Up" to join the STAC mail list and (optionally) register to access materials on the site. Click for terms.

STAC-ML Markets (Inference) Test Harness

This repository is used in STAC-ML™ Markets (Inference) benchmark tests and audits. It contains everything to run Inference Benchmarks EXCEPT a STAC Pack (an implementation for a given architecture), including:

  • The STAC-ML Markets (Inference) Benchmark Specifications Workbook (Excel)
  • Benchmark Overview (PDF)
  • Developer’s Guide (PDF)
  • Power-Monitoring Guide (PDF)
  • Test Execution Procedures (PDF)
  • Test Harness automation scripts (python and make)
  • Input Data Generation scripts (python)
  • Results analysis and visualization (iPython notebooks)

The Test Harness is accompanied by two additional repositories:

  1. A reference implementation
  2. The models used in the benchmark

If you have been previously permissioned to this software, please log in using your corporate email at
https://stacresearch.beanstalkapp.com/. Otherwise, please click the link below to request access.

Please log in to see file attachments. If you are not registered, you may register for no charge.