Custom scenarios

Instead of being simply step-based, custom scenarios are a full-fledged Python program which may execute everything that DSS scenarios can, while providing the user with full configurability and advanced logic capabilities.

What can a custom scenario do

A custom scenario can:

  • Execute all steps that can be defined in a “step-based scenario”. For more information, see Scenario steps.

  • Read metadata about the executed steps, like:

    • Failure / Success
    • Count and types of warnings
    • Detailed list of built datasets, trained models, …
  • Read details about trained models (performance metrics, …)

  • Read detailed parameters of the trigger that initiated this scenario.

  • Activate new versions of trained models

  • Read metrics and checks status for datasets

  • Read the build state: when was a dataset last built, a model last trained, …

  • Send custom messages through the reporters at any point of the scenario (not only at the beginning or end)

In addition, a custom scenario can use the whole internal and public Python API

The details of the scenarios API can be found in /python-api/plugins/custom_scenarios

Examples

Basic usage

This custom scenario builds datasets and trains models. Note that a step-based scenario suffices for this usage.

from dataiku.scenario import Scenario

# Create the main handle to interact with the scenario
scenario = Scenario()

# Build a dataset
scenario.build_dataset("mydatasetname")

# Build a partitioned data (this uses the partitions spec syntax)
scenario.build_dataset("mydatasetname", partitions="partition1,partition2")

# Train a model. The model id can be found in the URL of the model settings page
scenario.train_model("epae130z")

Send custom reports

Reports can be sent at any time of the scenario. You’ll need to have a preconfigured messaging channel.

from dataiku.scenario import Scenario
scenario = Scenario()

message_sender = scenario.get_message_sender("channel-name")

# You can then call send() with message params.
# params are specific to each message channel types

# SMTP mail example
message_sender.send(sender="", recipient="", subject="", message="")

# You can also call set_params to set params on the sender that will be reused for all subsequent 'send' calls
message_sender.set_params(sender="[email protected]", recipent="[email protected]")
message_sender.send(subject="All is well", message="Scenario is working as expected")

# Twilio SMS alert example
message_sender.send(fromNumber="", toNumber="", message="")