Limitations and supported versions¶
The MLflow model import feature is supported, and Dataiku tests it with a variety of different MLflow models. Dataiku makes best effort to ensure that the advanced capabilities of its MLflow import support are compatible with the widest possible variety of MLflow models.
However, MLflow imposes extremely few constraints on models, and different MLflow models are allowed to behave in arbitrary non-standard ways and to return completely different kind of data.
It is thus not possible to guarantee unfettered ability to use all features (notably advanced features such as performance evaluation, model comparison or drift analysis) with all types of models.
This page lists known compatibilities and incompatibilities
Import and scoring¶
The MLflow model import capability and scoring recipes in “direct output” mode are usually compatible with MLflow models supporting the “pyfunc” variant.
As of May 2022, this has been tested with:
Python Function
H2O
Keras
Pytorch
Scikit-learn
TensorFlow 1 and 2
ONNX
MXNet Gluon
XGBoost
LightGBM
Catboost
Spacy
FastAI
Statsmodels
Prophet
Dataiku does not support R MLflow models nor Spark Mlflow models
The import of MLflow models in DSS was successfully tested with MLflow versions 1.20.2 and 1.30.0. Later versions may also work, but modifications of implementation details of MLflow may cause bugs. Version 1.30.0 should generally be preferred, except for some older packages, such as TensorFlow 1 and Keras, whose support was removed in MLflow 1.21.0.
The following ML packages are supported in the versions specified in the below table. When using one of these packages, you should create a code environment including:
dataiku core packages
the specified version(s) of the ML package(s)
scikit-learn==1.0.2
statsmodels==0.13.5
protobuf==3.16.0
ML package |
ML packages versions |
MLflow version |
CatBoost |
catboost==0.26.1 |
mlflow==1.30.0 |
fast.ai 1 |
fastai==1.0.60 |
mlflow==1.20.2 |
fast.ai 2 |
fastai==2.7.10 |
mlflow==1.30.0 |
LightGBM |
lightgbm>=3.0,<3.1 |
mlflow==1.30.0 |
ONNX |
onnx==1.12.0 onnxruntime==1.12.0 (compatible ML package) |
mlflow==1.30.0 |
PyTorch |
torch==1.13.0 torchvision==0.14 torchmetrics==0.11.0 pytorch-lightning==1.8.2 |
mlflow==1.30.0 |
TensorFlow 1 / Keras 2.3 |
tensorflow==1.15.0 keras==2.3.1 keras-preprocessing==1.1.0 h5py==2.10.0 |
mlflow==1.20.2 |
TensorFlow 2 / Keras 2.10 |
tensorflow==2.10.0 tensorflow-estimator==2.10.0 keras==2.10.0 |
mlflow==1.30.0 |
XGBoost |
xgboost==1.5.0 |
mlflow==1.30.0 |
mlflow==1.30.0,
scikit-learn==1.0.2,
statsmodels==0.13.5,
protobuf==3.16.0,
torch==1.13.0,
torchvision==0.14,
torchmetrics==0.11.0,
pytorch-lightning==1.8.2
mlflow==1.30.0,
scikit-learn==1.0.2,
statsmodels==0.13.5,
protobuf==3.16.0,
tensorflow==2.10.0,
tensorflow-estimator==2.10.0,
keras==2.10.0
Evaluation¶
DSS also features the evaluation of regression or classification models on tabular input data. As of May 2022, this feature has been successfully tested in the following circumstances.
Please note that this is not a guarantee that you would necessarily be able to do the same, due to the high variability of models that can be saved (even within a single framework).
In all cases, the prediction_type should be set when importing the model. Please see below supported prediction types for the supported ML packages:
ML package |
binary classification |
multiclass classification |
regression |
CatBoost |
✓ |
✓ |
✓ |
fast.ai 1 |
✗ |
✗ |
✗ |
fast.ai 2 |
✗ |
✗ |
✓ |
LightGBM |
✓ |
✓ |
✓ |
ONNX |
✓ |
✓ |
✓ |
PyTorch |
✓ |
✓ |
✓ |
scikit-learn 1.0.2 |
✓ |
✓ |
✓ |
TensorFlow 1 / Keras 2.3 |
✗ |
✗ |
✓ |
TensorFlow 2 / Keras 2.10 |
✓ |
✓ |
✓ |
XGBoost |
✓ |
✓ |
✓ |