Customization of base images¶
Warning
This requires knowledge of Docker concepts and skills in creating custom Dockerfiles.
When building the base image with
./bin/dssadmin build-base-image --type container-exec
a base image is created with:
Python 2.7 and Python 3.6
R
No CUDA support
Building an image with CUDA support¶
The base image that is built by default (see setup) does not have CUDA support and cannot use NVidia GPUs.
You need to build a CUDA-enabled base image. To enable CUDA add the --with-cuda
option to the command line:
./bin/dssadmin build-base-image --type container-exec --with-cuda
We recommend that you give this image a specific tag using the --tag
option and keep the default base image “pristine”. We also recommend that you add the DSS version number in the image tag.
./bin/dssadmin build-base-image --type container-exec --with-cuda --tag dataiku-container-exec-base-cuda:X.Y.Z
where X.Y.Z is your DSS version number
Note
This image contains CUDA 10.0 and CuDNN 7.6. You can use
--cuda-version X.Y
to specify another DSS-provided version (9.0, 10.0, 10.1, 10.2, 11.0 and 11.2 are available). If you require other CUDA versions, you would have to create a custom image.Remember that depending on which CUDA version you build the base image (by default 10.0) you will need to use the corresponding tensorflow version.
Warning
After each upgrade of DSS, you must rebuild all base images and update code envs.
Multiple base images¶
If you don’t use the --tag
flag, DSS builds a base image with this naming scheme:
dku-exec-base-DSS_INSTALL_ID : dss-DSS_VERSION
Where
DSS_INSTALL_ID is the identifier of the DSS installation, found in the
install.ini
file.DSS_VERSION is the version of DSS.
If you don’t specify anything in the “base image” field of the DSS containerized execution configuration, this tag will automatically be used.
You can build other base images by appending the --tag IMAGE_NAME:IMAGE_VERSION
flag to the ./bin/dssadmin build-base-image --type container-exec
command.
Setting a proxy¶
You can set the proxy to use to build with --http-proxy
and --no-proxy
to set the http_proxy
and no_proxy
environment variables.
Adding system packages¶
There are cases where you would want to install additional system packages, generally because they are required by your code environments.
For that, add --system-packages package1,package2,package3
Add a Dockerfile fragment¶
You may want to add custom Dockerfile commands. For that, use --dockerfile-prepend PATH_TO_FILE
or --dockerfile-append PATH_TO_FILE
.
The prepended Dockerfile is added just after the FROM. The appended Dockerfile is added at the very end of the Dockerfile.
To add a file to the build context, to make the file available to use in Dockerfile commands added via fragment, use --copy-to-buildenv absolute/path/file.name file.name
.
Completely custom Dockerfile¶
For cases not covered, the generic processs would be:
Build a base image with the regular DSS mechanisms.
Write a custom Dockerfile that starts from the built base image, and add the required package.
Build this custom Dockerfile, and output a custom tag.
Enter this custom tag in the DSS containerized execution configuration.
Warning
After each upgrade of DSS, you must rebuild all base images, including custom ones.