Using unmanaged GKE clusters

Setup

Create your GKE cluster

To create a Google Kubernetes Engine (GKE) cluster, follow the Google Cloud Platform (GCP) documentation on creating a GKE cluster. We recommend that you allocate at least 16GB of memory for each cluster node. More memory may be required if you plan on running very large in-memory recipes.

You’ll be able to configure the memory allocation for each container and per-namespace in Dataiku DSS using multiple containerized execution configurations.

Prepare your local gcloud, docker, and kubectl commands

Follow the GCP documentation to ensure the following on your local machine (where DSS is installed):

  • The gcloud command has the appropriate permission and scopes to push to the Google Artifact Registry (GAR) service.

  • The kubectl command is installed and can interact with the cluster. This can be achieved by running the gcloud container clusters get-credentials your-gke-cluster-name command.

  • The docker command is installed, can build images and push them to GAR. The latter can be enabled by running the gcloud auth configure-docker command.

Note

Cluster management has been tested with the following versions of Kubernetes:

  • 1.23

  • 1.24

  • 1.25

  • 1.26

  • 1.27

  • 1.28

  • 1.29

  • 1.30

  • 1.31

  • 1.32

  • 1.33

  • 1.34

  • 1.35

  • 1.36

There is no known issue with other Kubernetes versions.

Create base images

Build the base image by following these instructions.

Create new image build and containerized execution configurations

  • Go to: Administration > Settings > Containerized Execution

  • Add a new image build configuration

  • Configure the GAR repository URL to use, e.g. <region>-docker.pkg.dev/my-gcp-project/my-repository

  • Set the GAR-specific push/authentication option on the image build configuration.

  • Add a new container execution configuration of type “Kubernetes”

  • Finish by clicking Push base images.

You’re now ready to run recipes and ML models in GKE.

Using GPUs

GCP provides GPU-enabled instances with NVIDIA GPUs. Using GPUs for containerized execution requires the following steps.

Enable GPU support on the cluster

Follow the GCP documentation on how to create a GKE cluster with GPU accelerators. You can also create a GPU-enabled node pool in an existing cluster.

Be sure to run the “DaemonSet” installation procedure, which needs several minutes to complete.

Add a custom reservation

For your containerized execution task to run on nodes with GPUs, and for GKE to configure the CUDA driver on your containers, the corresponding pods must be created with a custom limit (in Kubernetes parlance). This indicates that you need a specific type of resource (standard resource types are CPU and memory).

You must configure this limit in the containerized execution configuration. To do this:

  • In the “Custom limits” section, add a new entry with key nvidia.com/gpu and value 1 (to request 1 GPU).

  • Add the new entry and save your settings.

Deploy

You can now deploy your GPU-based recipes and models.