Cloudera (ex-Hortonworks) HDP¶

Warning

Support for HDP in DSS is Deprecated

All versions of HDP are past their End Of Support date by Cloudera. Users are strongly encouraged to migrate to CDP or Elastic AI infrastructure

DSS has deprecated support for HDP:

3.0.0
3.1.0
3.1.4
3.1.5

HDP 3.1 support¶

Because of a bug in Hive (https://jira.apache.org/jira/browse/HIVE-20568), the Hive dataset: cannot be used when using HDP 3.1 if the Hive database name contains an underscore.

The JDBC driver of Hive will fail to properly consider that the table exists and DSS will consequently always complain that the dataset is not ready.

Limitations¶

Validation of Hive recipes with “UNION” or “UNION ALL” statements is not possible with HDP 3.X

Security¶

Connecting to secure clusters is fully supported
User Isolation Framework is supported with Ranger

DSS regular security and Ranger¶

When using DSS in regular security mode to connect to a secure cluster with Ranger, you need to add a Ranger HDFS rule, as described in Hive.

DSS User Isolation Framework and Ranger¶

When using DSS in User Isolation Framework mode to connect to a secure cluster with Ranger, you need to do some configuration adjustments, as described in Hive.

Migrating to HDP 3.X¶

If adding HDP 3.X support to an already-configured DSS instance, you must add the following lines to all of your Spark configurations:

spark.sql.hive.metastore.jars = builtin
spark.sql.hive.metastore.version =  1.2.1

See Spark configurations for more information on setting up Spark configurations