NVIDIA NIM¶
Integrating NVIDIA NIM LLMs into Dataiku allows you to leverage high-performance inference for both hosted and self-hosted models. This capability is enabled via the NVIDIA NIM plugin, which provides both the LLM Mesh connectivity and management tools for Kubernetes-based deployments.
Capabilities¶
NVIDIA NIM integration provides two primary features to your Dataiku instance:
Dataiku LLM Mesh Connection¶
A specialized LLM Mesh connection with:
Access to NVIDIA NIM Text/Multimodal Chat Completion and Embedding models via the Dataiku LLM Mesh.
Support for streaming and tool calling for compatible NIM models.
This connection is infrastructure-agnostic; models can be hosted in the NVIDIA Cloud, self-hosted using the provided deployment macro, or managed on external infrastructure.
NIM Deployment Macro¶
For teams self-hosting on Kubernetes, a complementary macro is available to manage the underlying infrastructure directly from Dataiku:
Deploy, list, and remove the NVIDIA GPU Operator and NIM Operator.
Deploy, list, and remove NVIDIA NIM Services.
Note
Using the macro to deploy the GPU and NIM Operators is optional. In many environments, it may be preferable to manage these operators externally (e.g., via the OpenShift OperatorHub) before connecting them to Dataiku.
Prerequisites¶
General Requirements:
Dataiku >= v13.4.0
An NVIDIA AI Enterprise corporate subscription or an NVIDIA Developer Program membership.
The NVIDIA NIM Plugin installed from the Dataiku Plugin Store.
NIM Deployment Requirements (for self-hosting):
NIM Container Registry and NIM Model Repository credentials (these can be an NVIDIA Authentication API Key if using NGC).
- An attached Kubernetes cluster with:
An auto-provisioning Storage Class that supports “ReadWriteMany” access mode (see NIM Operator docs).
(Optional) The Prometheus operator installed on the cluster, required if leveraging horizontal pod autoscaling.
(Optional) The Nginx ingress controller installed on the cluster; without this, the deployment macro defaults to exposing the NIM Services using a NodePort Kubernetes service.
Configuration and Setup¶
To begin, install the NVIDIA NIM plugin (see the plugin installation guide). Once installed, configure the necessary credentials via Plugin Presets.
NIM API Keys (Hosted)¶
This preset stores per-user NIM API keys. Use this preset when the NIM LLM endpoints require API key authentication, such as when using NIM hosted on NVIDIA Build.
Navigate to Plugins > Installed > NVIDIA NIM Plugin > Settings > NIM API Keys > Add a Preset.
Every user can then set up the preset under their Dataiku profile > API Credentials > Credentials > Plugin credentials. Click the Edit icon and paste your NIM API key.
Self-hosted Credentials¶
This preset stores the NIM Container Registry and NIM Model Repository credentials required when self-hosting NIMs on an attached Kubernetes cluster using the NIM Deployment Macro.
Navigate to Plugins > Installed > NVIDIA NIM Plugin > Settings > Self-hosted credentials > Add a Preset.
For the Docker container registry, enter the host, username, and API key.
- For the NIM model registry:
If using NGC, simply enter your NGC API key.
If using an alternative model registry (such as JFrog), select the override model registry checkbox and enter the model registry host, protocol, and API key.
NIM Environment Variables¶
This preset provides a mechanism to override the values of NIM environment variables. It should only be used when self-hosting NIMs on an attached Kubernetes cluster using the NIM Deployment Macro.
Navigate to Plugins > Installed > NVIDIA NIM Plugin > Settings > NIM Environment Variables > Add a Preset.
Use this to override the value of any standard NIM environment variable.
NIM LLM Mesh Connection¶
Once your credentials are configured, you can define the connection to your NIM models:
Navigate to Administration > Connections > New Connection > Custom LLM (LLM Mesh).
Provide a connection name and select NVIDIA NIM Connector in the Plugin dropdown.
- To add models, click Add Model and configure the following fields:
Id: A unique name to identify the model.
Capability: Select Chat Completion or Text Embedding.
Type: NVIDIA NIM LLM Connector.
Keys Preset: Select the preset name defined in the plugin settings.
- Endpoint URL: Provide the complete URL for the service:
For chat completion:
https://<hostname>/v1/chat/completionsFor embeddings:
https://<hostname>/v1/retrieval/nvidia/embeddings
Model Key: Provide the model key accepted by NIM containers (e.g.,
google/gemma-2borNV-Embed-QA).Input Type: (Embedding models only) Set to query or passage (defaults to query).
Once the setup is complete, you can access NIM models both in LLM-powered Visual Recipes, Prompt Studios, and via the Python and REST LLM Mesh APIs.
NIM Service Deployment¶
If you are self-hosting, the NIM deployment macros are located in the Administration > Clusters > [Cluster Name] > Actions tab of the Kubernetes cluster.
GPU and NIM Operators¶
The first three macro actions provide the option to list, deploy, and remove the NVIDIA GPU and NIM Operators. If these Operators are not already available on the cluster, they must be deployed prior to deploying your first NIM Service.
NIM Services¶
The remaining macro actions provide the option to list, deploy, and remove NVIDIA NIM Services. Under the hood, Dataiku leverages the NVIDIA NIM Operator, so all options presented in the UI are those described in the NIM Operator documentation.
To deploy a NIM Service, first use the “NIM Service: Add” macro:
Then, use the “NIM Service: Inspect” macro to check the deployment status and retrieve the model endpoint once the Service has been successfully rolled out. The model endpoint can then be used to create a NIM LLM Mesh connection.
Case Study: Advancing Healthcare with Dataiku and NVIDIA¶
Curious how the Dataiku and NVIDIA partnership is advancing healthcare? Check out our blog post: