Spark Resource Manager Comparison of IBM Platform Conductor for Spark, Apache YARN and Apache Mesos – Phase 1

Stack under test:

  • IBM Platform Conductor 1.1 vs Apache YARN 2.6.3 vs Apache Mesos 0.26.0
  • Spark v1.5.2 with HDFS 2.6.3
  • Red Hat Enterprise Linux 7.1
  • 11 x Lenovo x 3630 M4 servers, 14 x 7200 RPM drives
  • 2 x 8-core Intel Xeon E5-2450 @ 2.10GHz
  • Mellanox MT27500 ConnectX-3 10GbE Adapters
  • IBM BNT RackSwitch G81240E 10GbE Switch

Please log in to see file attachments. If you are not registered, you may register for no charge.

Big data has become a big topic at STAC. All it takes to confirm this is a quick glance at discussions and presentations at STAC Summits over the past few years (particularly NY and London).