Running Hugging Face models

The LLM Mesh supports locally-running Hugging Face models for several tasks:

• Text generation

• Text embedding

• Text reranking

• Image generation

• Image embedding

• Other task-specific models

Dataiku Agent Management provides presets for some common Hugging Face models for easier setup. Compatible Hugging Face models can also always be configured manually.

• Setup & prerequisites
• Text generation
• Text embedding
• Text reranking
• Other model types
• Monitoring model instances
• The model cache

Caution

Running local large-scale Hugging Face models is a complex and costly setup, and both quality and performance often remain below proprietary LLM APIs. For first experiments with LLMs, Dataiku recommends hosted LLM APIs.

Furthermore, it usually requires setting up Kubernetes clusters, which is not a typical setup for Dataiku Agent Management.