Google Cloud Translation

Translate a text column to another language using Google Cloud Translation.

This capability is provided by the Google Cloud Translation plugin, which you need to install. Please see Installing plugins.

Note that the Google Cloud Translation API is a paid service, check their API pricing page for more information.

How to set up

If you are a Dataiku and Google Cloud admin user, follow these configuration steps right after you install the plugin. If you are not an admin, you can forward this to your admin and scroll down to the How to use section.

Get a service account key for the Translation API – in Google Cloud Console

You can follow the step-by-step instructions on this Google Cloud documentation page. Make sure that billing is activated and the Cloud Translation API is enabled on your Google Cloud project.

Create a service account and download a service account key as a JSON file.

Create an API configuration preset – in Dataiku

In Dataiku, open the Google Cloud Translation plugin, navigate to Settings > API configuration and create your first preset.

Configure the preset – in Dataiku

  • Fill the AUTHENTICATION settings

    • Copy-paste the content of your service account key from the previous step in the GCP service account key field. Make sure the key is valid JSON.

    • Alternatively, you may leave the field empty to use Application Default Credentials from the server environment. If you choose this option, follow Google’s authentication documentation.

  • (Optional) Review the API QUOTA and PARALLELIZATION settings

    • The Rate limit parameter sets the maximum number of requests per recipe during the Period, which is measured in seconds. The defaults are 6000 requests per 60 seconds. Calls are retried after a delay when this limit is reached.

    • You may need to decrease the Rate limit parameter if you envision that multiple recipes will run concurrently to call the API. For instance, if you want to allow 10 concurrent Dataiku activities, you can set this parameter at 6000/10 = 600 requests per minute.

    • The default Concurrency parameter means that 4 threads will call the API in parallel. This parallelization operates within the API Quota settings defined above.

  • Set the Permissions of your preset

    • You can declare yourself as the Owner of this preset and make it available to everybody, or to a specific group of users.

    • Any user belonging to one of these groups on your Dataiku instance will be able to see and use this preset.

Your preset is now ready to be used.

Later, you (or another Dataiku admin) will be able to add more presets. This can be useful to segment plugin usage by user group. For instance, you can create a “Default” preset for everyone and a “High performance” one for your Marketing team. To use separate billing for each team, configure credentials for separately billed Google Cloud projects.

How to use

Let’s assume that you have installed this plugin and that you have a Dataiku project with a dataset containing a column of text to translate.

Google Cloud Translation recipe

To create your first recipe, navigate to the Flow, click on the + RECIPE button and access the Natural Language Processing menu. If your dataset is selected, you can directly find the plugin in the right panel.

Select the input dataset and create an output dataset for the translated text.

Input

Dataset with a text column to translate.

Settings

  • Review INPUT parameters

    • The Text column parameter is the column in the input dataset that you wish to translate.

    • The Source language parameter is the original language of the Text column. Select Auto-detect to infer the original language.

    • The Target language parameter is the language you would like to translate to. Choose from the languages available in the recipe.

    • The Format parameter is the formatting your input text is in. Leave the default to Text if you have plain text or change to HTML.

  • Review CONFIGURATION parameters

    • Select an API configuration preset made available by your Dataiku admin. This parameter is required.

    • The Fail on error parameter lets you choose whether API errors abort execution. If unchecked, handled API errors are logged in two additional columns in the output. Configuration errors can still cause the recipe to fail.

Run the recipe to populate the output dataset.

Output

Dataset with text translated to another language

Generated column names receive a numeric suffix if they conflict with existing input column names. Empty, whitespace-only, or non-string values are not sent to the API and produce empty translation and detected-language values.