Azure Translation

DSS can translate a text column to another language using Azure Translator.

This capability is provided by the Azure Translation plugin, which you need to install. Please see Installing plugins.

Note that the Azure Translation API is a paid service, check their API pricing page for more information.

How to set up

If you are a Dataiku and Microsoft Azure admin user, follow these configuration steps right after you install the plugin. If you are not an admin, you can forward this to your admin and scroll down to the How to use section.

Create an Azure Translator Resource

In order to use the Azure Translation API, you need an Azure account. If you don’t have one yet, you can sign up for a free account here.

Next you will have to create an Azure Translator resource. Follow the official instructions here to create an Azure Translator resource.

Once created, navigate to the Translator resource and under “Keys and Endpoint” find your API Key(s) and Location.

Create an API configuration preset – in Dataiku

In Dataiku, open the Azure Translation plugin, navigate to Settings > API configuration and create your first preset.

Configure the preset – in Dataiku

• Fill the AUTHENTICATION settings

Fill in the required Azure Translator API key and Azure Translator location fields with one of your resource’s API keys and its region, for example francecentral.

• (Optional) Review the PARALLELIZATION and ATTEMPTS settings

  • The default Concurrency parameter means that 4 API requests can run in parallel.

  • The default Maximum Attempts is 3 attempts in total, including the initial request.

    • Failed API requests returning HTTP errors (for example, authentication or throttling errors) are retried up to this limit. Other exceptions are not retried.

    • Note that Azure may charge you depending on the nature of the error, for each additional attempt.

  • The Waiting Interval specifies how many seconds to wait before retrying a failed attempt, with a default of 5 seconds. In case of a throttling exception due to too many requests increasing the Waiting Interval may help, however, we recommend first decreasing the Concurrency setting.

• Set the Permissions of your preset

  • You can declare yourself as the Owner of this preset and make it available to everybody, or to a specific group of users.

  • Any user belonging to one of these groups on your Dataiku instance will be able to see and use this preset.

Your preset is now ready to be used.

Configuring additional presets can be useful to segment plugin usage by user group. For instance, you can create a “Default” preset for everyone and a “High performance” one for your Marketing team, with separate billing for each team.

How to use

Let’s assume that you have installed this plugin and that you have a Dataiku project with a dataset containing a column of text to translate.

Azure Translation recipe

To create your first recipe, navigate to the Flow, click on the + RECIPE button and access the Natural Language Processing menu. If your dataset is selected, you can directly find the plugin in the right panel.

Select the Azure Translation recipe, choose an input dataset and create an output dataset. Configure the settings below, then run the recipe.

Input

Dataset with a string column to translate.

Settings

• Review INPUT parameters

  • The Text column parameter is the column in the input dataset that you wish to translate.

  • The Source language parameter is the original language of the Text column. If you would like Azure Translator to infer the original language, select Auto-detect. Choose source and target languages from the recipe’s dropdown lists; these may contain fewer languages than the Azure language support list.

  • The Target language parameter is the language you would like to translate to.

• Review CONFIGURATION parameters

  • The required API configuration preset parameter lets you select a preset made available to you by your Dataiku admin.

  • The Fail on error parameter lets you choose if the recipe should abort when an API request fails after retries. If unchecked, API HTTP errors are logged in two additional columns in the output and the translation is empty for those rows. Other exceptions can still cause the recipe to fail.

Output

• Dataset with text translated to another language

The columns of the output dataset are as follows:

Column	Description
[Input dataset columns]	All columns from the input dataset will be preserved
[selected column]_language	The detected language name of the selected column. Only present if Auto-detect has been selected as the source language
[selected column]_[target language code]	The selected column in its translated version. Hyphens in the language code are replaced with underscores, for example text_pt_PT
translation_api_response	Raw API response in JSON form
translation_api_error_message	The error message in case an API HTTP error occurred
translation_api_error_type	The error type in case an API HTTP error occurred

Generated column names receive a numeric suffix if needed to avoid overwriting an input column. Empty, whitespace-only or missing text produces an empty translation and, with Auto-detect, an empty detected language; no API request is made for these rows.