Limitations and supported versions¶
Import and scoring¶
Warning
While Dataiku makes best effort to ensure that its MLflow import support is compatible with the widest possible variety of MLflow models, MLflow imposes extremely few constraints on models, and so it is not possible to guarantee unfettered ability to use all features with all types of models.
The MLflow model import capability and scoring recipes in “direct output” mode should be compatible with MLflow models supporting the “pyfunc” variant.
As of May 2022, this means:
Python Function
H2O
Keras
Pytorch
Scikit-learn
TensorFlow 1 and 2
ONNX
MXNet Gluon
XGBoost
LightGBM
Catboost
Spacy
FastAI
Statsmodels
Prophet
The import of MLflow models in DSS was successfully tested with MLflow versions 1.20.2 and 1.30.0. Later versions may also work, but modifications of implementation details of MLflow may cause bugs. Version 1.30.0 should generally be preferred, except for some older packages, such as TensorFlow 1 and Keras, whose support was removed in MLflow 1.21.0.
The following ML packages are supported in the versions specified in the below table. When using one of these packages, you should create a code environment including:
dataiku core packages
the specified version(s) of the ML package(s)
scikit-learn==1.0.2
statsmodels==0.13.5
protobuf==3.16.0
ML package |
ML packages versions |
MLflow version |
CatBoost |
catboost==0.26.1 |
mlflow==1.30.0 |
fast.ai 1 |
fastai==1.0.60 |
mlflow==1.20.2 |
fast.ai 2 |
fastai==2.7.10 |
mlflow==1.30.0 |
LightGBM |
lightgbm>=3.0,<3.1 |
mlflow==1.30.0 |
ONNX |
onnx==1.12.0 onnxruntime==1.12.0 (compatible ML package) |
mlflow==1.30.0 |
PyTorch |
torch==1.13.0 torchvision==0.14 torchmetrics==0.11.0 pytorch-lightning==1.8.2 |
mlflow==1.30.0 |
TensorFlow 1 / Keras 2.3 |
tensorflow==1.15.0 keras==2.3.1 keras-preprocessing==1.1.0 h5py==2.10.0 |
mlflow==1.20.2 |
TensorFlow 2 / Keras 2.10 |
tensorflow==2.10.0 tensorflow-estimator==2.10.0 keras==2.10.0 |
mlflow==1.30.0 |
XGBoost |
xgboost==1.5.0 |
mlflow==1.30.0 |
mlflow==1.30.0,
scikit-learn==1.0.2,
statsmodels==0.13.5,
protobuf==3.16.0,
torch==1.13.0,
torchvision==0.14,
torchmetrics==0.11.0,
pytorch-lightning==1.8.2
mlflow==1.30.0,
scikit-learn==1.0.2,
statsmodels==0.13.5,
protobuf==3.16.0,
tensorflow==2.10.0,
tensorflow-estimator==2.10.0,
keras==2.10.0
Evaluation¶
DSS also features the evaluation of regression or classification models on tabular input data. As of May 2022, this feature has been successfully tested in the following circumstances.
Please note that this is not a guarantee that you would necessarily be able to do the same, due to the high variability of models that can be saved (even within a single framework).
In all cases, the prediction_type should be set when importing the model. Please see below supported prediction types for the supported ML packages:
ML package |
binary classification |
multiclass classification |
regression |
CatBoost |
✓ |
✓ |
✓ |
fast.ai 1 |
✗ |
✗ |
✗ |
fast.ai 2 |
✗ |
✗ |
✓ |
LightGBM |
✓ |
✓ |
✓ |
ONNX |
✓ |
✓ |
✓ |
PyTorch |
✓ |
✓ |
✓ |
scikit-learn 1.0.2 |
✓ |
✓ |
✓ |
TensorFlow 1 / Keras 2.3 |
✗ |
✗ |
✓ |
TensorFlow 2 / Keras 2.10 |
✓ |
✓ |
✓ |
XGBoost |
✓ |
✓ |
✓ |