Instead of being simply step-based, custom scenarios are a full-fledged Python program which may execute everything that DSS scenarios can, while providing the user with full configurability and advanced logic capabilities.
What can a custom scenario do¶
A custom scenario can:
Execute all steps that can be defined in a “step-based scenario”. For more information, see Scenario steps.
Read metadata about the executed steps, like:
Failure / Success
Count and types of warnings
Detailed list of built datasets, trained models, …
Read details about trained models (performance metrics, …)
Read detailed parameters of the trigger that initiated this scenario.
Activate new versions of trained models
Read metrics and checks status for datasets
Read the build state: when was a dataset last built, a model last trained, …
Send custom messages through the reporters at any point of the scenario (not only at the beginning or end)
In addition, a custom scenario can use the whole internal and public Python API.
The details of the scenarios API can be found in Scenarios.
The scenario API cannot be used in a Python recipe/Jupyter notebook.
This custom scenario builds datasets and trains models. Note that a step-based scenario suffices for this usage.
from dataiku.scenario import Scenario # Create the main handle to interact with the scenario scenario = Scenario() # Build a dataset scenario.build_dataset("mydatasetname") # Build a partitioned data (this uses the partitions spec syntax) scenario.build_dataset("mydatasetname", partitions="partition1,partition2") # Train a model. The model id can be found in the URL of the model settings page scenario.train_model("epae130z")
Send custom reports¶
Reports can be sent at any time of the scenario. You’ll need to have a preconfigured messaging channel.
from dataiku.scenario import Scenario scenario = Scenario() message_sender = scenario.get_message_sender("channel-name") # You can then call send() with message params. # params are specific to each message channel types # SMTP mail example message_sender.send(sender="", recipient="", subject="", message="") # You can also call set_params to set params on the sender that will be reused for all subsequent 'send' calls message_sender.set_params(sender="[email protected]", recipent="[email protected]") message_sender.send(subject="All is well", message="Scenario is working as expected") # Twilio SMS alert example message_sender.send(fromNumber="", toNumber="", message="")
You can also send message to Microsoft Teams. Here is an example where
my-teams-channel is a preconfigured messaging channel:
from dataiku.scenario import Scenario scenario = Scenario() message_sender = scenario.get_message_sender("my-teams-channel") message_sender.send(message="Scenario is working as expected")
Here is another example that uses an ad-hoc channel:
from dataiku.scenario import Scenario scenario = Scenario() message_sender = scenario.get_message_sender(None, "msft-teams-scenario") message_sender.send(message="Scenario is working as expected", webhookUrl="Webhook URL of the Teams channel")
If you want to send an email message in HTML format, you can pass the parameter
sendAsHTML=True in your send function. Here is an example where
my-email-channel is a preconfigured messaging channel:
from dataiku.scenario import Scenario scenario = Scenario() message_sender = scenario.get_message_sender("my-email-channel") # some HTML-formatted text html_message = "<p style='color:orange'>Your scenario is running!</p><p>While waiting, check out this <a href='https://www.dataiku.com'>awesome website</a></p>" message_sender.send(subject="The scenario is doing well", message=html_message, sendAsHTML=True)