Setting up Hadoop and Spark integration

Data Science Studio is able to connect to a Hadoop cluster and to:

• Read and write HDFS datasets

• Run Hive queries and scripts

• Run Impala queries

• Run Pig scripts

• Run preparation recipes on Hadoop

In addition, if you setup Spark integration, you can:

• Run SparkSQL queries

• Run preparation, join, stack and group recipes on Spark

• Run PySpark & SparkR scripts

• Train & use Spark MLLib models

See Setting up Hadoop integration and Setting up Spark integration