The AMPLab-born and bred Apache Spark system continues to gain popularity as demonstrated by two recent announcements. MapR announced that they are incorporating the “complete Apache Spark stack” in their Hadoop distribution. Another major Hadoop vendor, Hortonworks, somewhat more subtly announced that they were including a “a tech preview of Apache Spark for distributed in-memory processing” in their next release of the Hortonworks Data Platform (HDP 2.1). These new announcements, combined with the fact that Cloudera has been including Spark in their distribution for some time now, are strong indicators of Spark’s growing role in the Big Data analytics mainstream.
National Science Foundation
Expeditions in Computing