STAC-A2 on AWS: Initial investigations

Initial price-performance comparisons of ten different EC2 instance types.

24 February 2015

As the first milestone in a planned series of investigations of infrastructure-as-a-service (IaaS) offerings, STAC has just completed an initial round of STAC-A2 testing on ten different Elastic Compute Cloud (EC2) instance types on Amazon Web Services (AWS), comparing their price-performance and absolute performance, as well as relating the latter to standalone systems we have tested. (Spoiler: there are some big differences in all of these comparisons.)

All of this information is available in the STAC Vault to premium subscribers, starting with the Tech Note that summarizes the comparisons.

The results and methodologies should be of interest to any organization considering internal or external cloud deployments of compute-/memory-bound analytics or just looking for industry standard benchmarks of price-performance for this sort of high-performance financial workload.

The Tech Note links to STAC Reports, STAC Configuration Disclosures, and other detailed information for each instance tested. It also describes our experiences trying to configure and optimize some of these EC2 instance types, particularly the new Haswell-based C4 instances, with tips that may be helpful to others working to evaluate or deploy compute-bound apps on AWS (this includes an issue we uncovered that could affect floating-point performance for anyone using certain configurations of virtualization under Red Hat in any private or public environment).

If your firm does not have a paid subscription to STAC-A2 research, this is a great time to get on board. In addition to this AWS study and previous studies, you can look forward to more research and software, including:

• studies of additional cloud products
• a research program involving the Julia programming language
• a research program involving OpenCL on a variety of platforms
• implementations on new architectures as they emerge.

Premium subscribers have access to all of this research as well as the code used in each project plus the micro-detailed configuration information that led to each set of results. To learn about subscription options, please contact us.