Advanced topics¶
Security of Spark on Kubernetes¶
When running with User Isolation Framework, the Spark driver process runs as the impersonated end-user. Thus, the interaction between Spark and Kubernetes also runs as the impersonated end-user.
This requires that each impersonated end-user has credentials to access the Kubernetes. While this deployment is completely possible, it is not typically the case (each user needs to have a ~/.kube/config
file with proper credentials for the Kubernetes cluster).
To make it easier to run Spark on Kubernetes with User Isolation Framework, DSS features a “managed Spark on Kubernetes” mode. For details and setup examples, please see our reference architecture.