Text cleaning

Text cleaning is the process of cleaning up, simplifying text, and preparing it for further analysis

Dataiku provides offline text cleaning

Offline text cleaning

The native text cleaning capability of Dataiku provides capabilities in 59 languages

It provides:

• Tokenization

• Filtering of punctuation, stop words, and multiple other categories

• Lemmatization

It is an offline capability, meaning that it does not leverage a 3rd party API.

This capability is provided by the Text Preparation plugin, which you need to install. Please see Installing plugins.

For detailed recipe settings and supported languages, see Text Preparation.