Remote Docker daemons

The docker command line is the Docker client. The Docker daemon, responsible for building images and running the containers, may be on the same server or may be remote.

Rationale

There are some use cases for a remote docker daemon running your containers:

  • Offload heavy work onto other servers
  • Leverage resources available on another machine (like GPUs)

Furthermore, the Docker daemon runs with high privileges and on some setups it may be moved to another server rather than kept locally.

Setup

See also Setting up.

With a registry

You do not need a specific setup if all of the following conditions are met:

  • You are using an image registry
  • You have on the DSS server a local Docker daemon that can push to that registry
  • The remote Docker daemon can pull from that registry

Then the local Docker daemon can build the images and the remote one can use them to run the containers.

../_images/remote-daemon-with-registry.svg

Without a registry

Otherwise, you need to have the remote Docker daemon build the images.

../_images/remote-daemon-without-registry.svg

You still need the local docker command (Docker client) to be fully functional and usable by the user running DSS. You then need to specify the Docker daemon host before building the base image:

export DOCKER_HOST=host_of_my_remote_daemon

If you are using TLS to securely connect to the remote daemon, you will also need the corresponding environment variables:

export DOCKER_TLS_VERIFY=1
export DOCKER_CERT_PATH=/path/to/docker/cert/directory/

DOCKER_CERT_PATH is the path to a folder that contains the client certificates: ca.pem, cert.pem and key.pem. It can be omitted if it is the default ~/.docker/. For more information about Docker and TLS, refer to the Docker documentation.

You can now build the base image as specified in Setting up.

You will then need to specify the same settings in the container execution configurations:

  • The Docker host
  • If using TLS authentication, check Enable TLS and provide the path to the directory with the certificates

If necessary, rebuild images for Code Environments; see Using code envs with container execution.

Usage

By selecting the corresponding container execution configuration, you are now ready to run deploy your worload on remote Docker containers.

Note

If you have several remote Docker daemons, you will need to create multiple container execution configurations, and manually dispatch execution between those configurations. DSS does not automatically dipatch between multiple configurations.