Customization of base images¶
This requires knowledge of Docker concepts and skills in creating custom Dockerfiles.
When building the base image with
./bin/dssadmin build-base-image --type container-exec
a base image is created with:
Python 2.7 and Python 3.6
No CUDA support
The base image that is built by default (see setup) does not have CUDA support and cannot use NVidia GPUs.
You need to build a CUDA-enabled base image. To enable CUDA add the
--with-cuda option to the command line:
./bin/dssadmin build-base-image --type container-exec --with-cuda
We recommend that you give this image a specific tag using the
--tag option and keep the default base image “pristine”. We also recommend that you add the DSS version number in the image tag.
./bin/dssadmin build-base-image --type container-exec --with-cuda --tag dataiku-container-exec-base-cuda:X.Y.Z
where X.Y.Z is your DSS version number
This image contains CUDA 10.0 and CuDNN 7.6. You can use
--cuda-version X.Yto specify another DSS-provided version (9.0, 10.0, 10.1, 10.2, 11.0 and 11.2 are available). If you require other CUDA versions, you would have to create a custom image.
Remember that depending on which CUDA version you build the base image (by default 10.0) you will need to use the corresponding tensorflow version.
After each upgrade of DSS, you must rebuild all base images and update code envs.
If you don’t use the
--tag flag, DSS builds a base image with this naming scheme:
dku-exec-base-DSS_INSTALL_ID : dss-DSS_VERSION
DSS_INSTALL_ID is the identifier of the DSS installation, found in the
DSS_VERSION is the version of DSS.
If you don’t specify anything in the “base image” field of the DSS containerized execution configuration, this tag will automatically be used.
You can build other base images by appending the
--tag IMAGE_NAME:IMAGE_VERSION flag to the
./bin/dssadmin build-base-image --type container-exec command.
You can set the proxy to use to build with
--no-proxy to set the
no_proxy environment variables.
There are cases where you would want to install additional system packages, generally because they are required by your code environments.
For that, add
You may want to add custom Dockerfile commands. For that, use
--dockerfile-prepend PATH_TO_FILE or
The prepended Dockerfile is added just after the FROM. The appended Dockerfile is added at the very end of the Dockerfile.
To add a file to the build context, to make the file available to use in Dockerfile commands added via fragment, use
--copy-to-buildenv absolute/path/file.name file.name.
For cases not covered, the generic processs would be:
Build a base image with the regular DSS mechanisms.
Write a custom Dockerfile that starts from the built base image, and add the required package.
Build this custom Dockerfile, and output a custom tag.
Enter this custom tag in the DSS containerized execution configuration.
After each upgrade of DSS, you must rebuild all base images, including custom ones.