Custom scenarios

Instead of being simply step-based, custom scenarios are a full-fledged Python program which may execute everything that DSS scenarios can, while providing the user with full configurability and advanced logic capabilities.

What can a custom scenario do

A custom scenario can:

  • Execute all steps that can be defined in a “step-based scenario”. For more information, see Scenario steps.

  • Read metadata about the executed steps, like:

    • Failure / Success

    • Count and types of warnings

    • Detailed list of built datasets, trained models, …

  • Read details about trained models (performance metrics, …)

  • Read detailed parameters of the trigger that initiated this scenario.

  • Activate new versions of trained models

  • Read metrics and checks status for datasets

  • Read the build state: when was a dataset last built, a model last trained, …

  • Send custom messages through the reporters at any point of the scenario (not only at the beginning or end)

In addition, a custom scenario can use the whole internal and public Python API.

The details of the scenarios API can be found in Scenarios.

Note

The scenario API cannot be used in a Python recipe/Jupyter notebook.

Examples

Basic usage

This custom scenario builds datasets and trains models. Note that a step-based scenario suffices for this usage.

from dataiku.scenario import Scenario

# Create the main handle to interact with the scenario
scenario = Scenario()

# Build a dataset
scenario.build_dataset("mydatasetname")

# Build a partitioned data (this uses the partitions spec syntax)
scenario.build_dataset("mydatasetname", partitions="partition1,partition2")

# Train a model. The model id can be found in the URL of the model settings page
scenario.train_model("epae130z")

Send custom reports

Reports can be sent at any time of the scenario. You’ll need to have a preconfigured messaging channel.

from dataiku.scenario import Scenario
scenario = Scenario()

message_sender = scenario.get_message_sender("channel-name")

# You can then call send() with message params.
# params are specific to each message channel types

# SMTP mail example
message_sender.send(sender="", recipient="", subject="", message="")

# You can also call set_params to set params on the sender that will be reused for all subsequent 'send' calls
message_sender.set_params(sender="[email protected]", recipent="[email protected]")
message_sender.send(subject="All is well", message="Scenario is working as expected")

# Twilio SMS alert example
message_sender.send(fromNumber="", toNumber="", message="")

You can also send message to Microsoft Teams. Here is an example where my-teams-channel is a preconfigured messaging channel:

from dataiku.scenario import Scenario
scenario = Scenario()

message_sender = scenario.get_message_sender("my-teams-channel")
message_sender.send(message="Scenario is working as expected")

Here is another example that uses an ad-hoc channel:

from dataiku.scenario import Scenario
scenario = Scenario()

message_sender = scenario.get_message_sender(None, "msft-teams-scenario")
message_sender.send(message="Scenario is working as expected", webhookUrl="Webhook URL of the Teams channel")

If you want to send an email message in HTML format, you can pass the parameter sendAsHTML=True in your send function. Here is an example where my-email-channel is a preconfigured messaging channel:

from dataiku.scenario import Scenario
scenario = Scenario()

message_sender = scenario.get_message_sender("my-email-channel")
# some HTML-formatted text
html_message = "<p style='color:orange'>Your scenario is running!</p><p>While waiting, check out this <a href='https://www.dataiku.com'>awesome website</a></p>"
message_sender.send(subject="The scenario is doing well", message=html_message, sendAsHTML=True)