Hortonworks HDP

DSS supports HDP 2.5 and 2.6, 3.0 and 3.1

HDP 3.X support

Warning

Experimental feature: Support for HDP 3.X is still experimental, with a best-effort support

  • When running DSS on HDP 3.X, the “Hive CLI” modes are not available for interaction with Hive. Only HiveServer2 mode is available. More details are available at DSS and Hive
  • If adding HDP 3.X support to an already-configured DSS instance, you must add the following lines to all of your Spark configurations:
spark.sql.hive.metastore.jars = builtin
spark.sql.hive.metastore.version =  1.2.1

See Spark configurations for more information on setting up Spark configurations

HDP 3.1 support

Because of a bug in Hive (https://jira.apache.org/jira/browse/HIVE-20568), the Hive dataset
cannot be used when using HDP 3.1 if the Hive database name contains an underscore.

The JDBC driver of Hive will fail to properly consider that the table exists and DSS will consequently always complain that the dataset is not ready.

Security

  • Connecting to secure clusters is fully supported
  • Multi-user security is supported with Ranger

DSS regular security and Ranger

When using DSS in regular security mode to connect to a secure cluster with Ranger, you need to add a Ranger HDFS rule, as described in DSS and Hive.

DSS multi-user-security and Ranger

When using DSS in multi-user security mode to connect to a secure cluster with Ranger, you need to do some configuration adjustements, as described in DSS and Hive.