Amazon Translation

You can translate a text column to another language using Amazon Translate.

This capability is provided by the Amazon Translation plugin, which you need to install. Please see Installing plugins.

Note that the Amazon Translation API is a paid service, check their API pricing page for more information.

How to set up

If you are a Dataiku and AWS (Amazon Web Services) admin user, follow these configuration steps right after you install the plugin. If you are not an admin, you can forward this to your admin and scroll down to the How to use section.

Create a privileged IAM User for Amazon Translation in the AWS Console

In order to use the Amazon Translation API, you need an AWS account. If you don’t have one yet, you can sign up for the free tier here.

Next, you need to create credentials with the necessary permissions using AWS Identity and Access Management (IAM). If you don’t have an IAM user yet, create one first.

Grant the user permission to call translate:TranslateText, directly or through a group. To use Auto-detect for the source language, also grant comprehend:DetectDominantLanguage and choose a region that supports Amazon Comprehend. See AWS permissions for Amazon Translate.

Make sure to take note of the Access key ID & Secret access key which will appear after creation. Once you have an IAM user with the necessary privileges, you just need to provide DSS with the credentials.

Create an API configuration preset – in Dataiku

In Dataiku DSS, open the Amazon Translation plugin, navigate to Settings > API configuration, and create your first preset.

Configure the preset – in Dataiku

• Fill the AUTHENTICATION settings

  • Enter your AWS access key id and AWS secret access key in the corresponding fields. Enter an AWS session token if using temporary credentials. Set AWS region name to the region code for the service, for example eu-west-1.

  • Alternatively, leave AWS access key id empty to use the AWS credentials and region configured on the server. In this mode, the other credential fields and AWS region name in the preset are ignored. Ensure that the recipe execution environment can access the server’s AWS configuration.

• (Optional) Review the PARALLELIZATION and ATTEMPTS settings

  • Concurrency defaults to 4 threads calling the API in parallel.

  • Maximum Attempts defaults to 5 and allows up to five retries after the initial API request for retryable errors, such as throttling or temporary service failures. Errors such as invalid credentials are not necessarily retried. See AWS SDK retry behavior.

    • Note that AWS may charge you depending on the nature of the error, for each additional attempt.

• Set the Permissions of your preset

  • You can declare yourself as the Owner of this preset and make it available to everybody, or to a specific group of users.

  • Any user belonging to one of these groups on your Dataiku DSS instance will be able to see and use this preset.

Your preset is now ready to be used.

Later, you (or another Dataiku admin) will be able to add more presets. This can be useful to segment plugin usage by user group. For instance, you can create a “Default” preset for everyone and a “High performance” one for your Marketing team, with separate billing for each team.

How to use

Let’s assume that you have installed this plugin and that you have a Dataiku project with a dataset containing a column of text to translate.

Amazon Translation recipe

To create your first recipe, navigate to the Flow, click on the + RECIPE button and access the Natural Language Processing menu. If your dataset is selected, you can directly find the plugin in the right panel.

Input

Dataset with a text column to translate.

Settings

• Review INPUT parameters

  • The Text column parameter is the column in the input dataset that you wish to translate.

  • The Source language parameter is the original language of the Text column. Select Auto-detect to infer it, subject to the permissions and region requirements above. Choose source and target languages from the recipe’s lists.

  • The Target language parameter is the language you would like to translate to.

• Review CONFIGURATION parameters

  • Select an API configuration preset made available by your Dataiku admin.

  • The Fail on error parameter lets you choose if the recipe should abort execution if any issues are raised. If unchecked, any errors will be logged in two additional columns in the output.

Select or create an output dataset, then run the recipe.

Output

Dataset with text translated to another language.

The columns of the output dataset are as follows:

Column	Description
[Input dataset columns]	All columns from the input dataset will be preserved
[selected column]_[target language code]	The translated text. Hyphens in the language code are replaced with underscores, for example text_zh_TW
translation_api_response	Raw API response in JSON form
translation_api_error_message	The error message in case an error occurred. Only present if Fail on error is not selected during configuration
translation_api_error_type	The error type in case an error occurred. Only present if Fail on error is not selected during configuration

If an added column name already exists in the input, a numeric suffix is appended to preserve the input column. Missing, non-text, and whitespace-only values are not sent to the API and produce an empty translation and an empty JSON response ({}).