Using managed AKS clusters

Initial setup

Create your ACR registry

If you already have an Azure Container Registry (ACR) up and ready, you can skip this section and go to Install the AKS plugin.

Otherwise, follow the Azure documentation on how to create your ACR registry.

Warning

We recommend that you pay extra attention to the Azure container registry pricing plan, as it is directly related to the registry storage capacity.

Install the AKS plugin

To use Microsoft Azure Kubernetes Service (AKS), begin by installing the “AKS clusters” plugin from the Plugins store in Dataiku DSS. For more details, see the instructions for installing plugins.

Prepare your local az, docker, and kubectl commands

Follow the Azure documentation to ensure the following on your local machine (where DSS is installed):

  • The az command is properly logged in. As of October 2019, this implies running the az login --service-principal --username client_d --password client_secret --tenant tenant_id command.
  • The docker command can successfully push images to the ACR repository. As of October 2019, this implies running the az acr login --name your-registry-name command.
  • The kubectl command can interact with the cluster. As of October 2019, this implies running the az aks get-credentials --resource-group your-rg --name your-cluster-name command.

Create base images

Build the base image by following these instructions.

Create a new containerized execution configuration

Go to Administration > Settings > Containerized execution, and add a new execution configuration of type “Kubernetes”.

  • In particular, to set up the image registry, the URL must be of the form your-registry-name.azurecr.io.
  • Finish by clicking Push base images.

Cluster configuration

  • The connection is where you define how to connect to Azure. This can be done either inline in each cluster (not recommended) or via a preset in the “AKS connection” plugin settings (recommended).
  • By default, the service principal associated to the cluster will be the same as the one used on the DSS machine. You can change this by enabling the Use distinct credentials option and defining a specific connection, either inline or via a preset.
  • Cluster nodes is where you define the number and type of nodes that you want in your cluster. You can define the properties of a node pool either inline (not recommended) or as a preset in the “Node pools” plugin settings (recommended). You have the possibility to define multiple node pools, each with its own properties.

Other

When you create a cluster using the AKS plugin, Microsoft will receive information that this cluster was provisioned from a Dataiku DSS instance. Microsoft is able to correlate the Azure resources that are used to support the software. Microsoft collects this information to provide the best experiences with their products and to operate their business. The data is collected and governed by Microsoft’s privacy policies, which can be found here.

To disable this, set a DISABLE_AZURE_USAGE_ATTRIBUTION environment variable to 1 in DATADIR/bin/env-site.sh.