WARN_RECIPE_SPARK_INDIRECT_HDFS: No direct access to read/write HDFS dataset¶
You are running a recipe that uses Spark (either a Spark code recipe, or a visual recipe using the Spark engine). This recipe either reads or writes a HDFS dataset.
However, your administrator has not granted you the “Read connection details” permission on the connection in which the HDFS dataset lies.
Without this permission, your Spark job will not be able to read (or write) the HDFS dataset directly, and will indeed need to go through a slow path. This will very strongly degrade the performance of your Spark job.