Reporting on scenario runs¶

Dataiku DSS provides the means to add reporters in a scenario. These reporters can be used to inform teams of users about scenario activities. For example, scenario reporters can update users about the training of models or changes in data quality. Reporters can also create actionable messages that users can receive within their email or through other messaging channels.

See our lesson on Reporting Scenario Activities for a tutorial on setting up reporters that send scenario updates to Microsoft Teams.

Executing scenario reporters¶

You can instruct a reporter to send messages at any point of a scenario run:

To send a message right before a scenario run starts, add a reporter to the scenario in the Settings pane, and set the value of “Send on scenario” to Start.
To send a message during the scenario run, add a Send message step (in a step-based scenario) or use get_message_sender() on the Scenario object (in a script scenario).
To send a message right after a scenario run ends, add a reporter to the scenario in the Settings pane, and set “Send on scenario” to End. You can further control whether the reporter runs at the end of the scenario, by specifying the “Run condition” as one of the expressions: outcome == 'SUCCESS' or outcome == 'FAILED' or outcome == 'ABORTED'. You can set up several reporters, each having a run condition and a corresponding message.

Building the message contents¶

Each reporter must define a message to broadcast on the selected messaging channel, and/or define some channel variables to pass to the reporter itself.

Reporting to mail provides the most versatility when building messages.

Mail reporter¶

Before you can send emails, an administrator needs to setup a “SMTP” channel in Administration > Settings > Notifications & Integrations.

Dataiku DSS sends messages by mail in either HTML or plain text format, and two engines are available to build the messages:

Freemarker-based templating
Variable-based formatting, using DSS formulas

In both cases, the results of the scenario run are offered to the engines as named variables.

DSS comes with a default Freemarker-based template, and changing the “Message source” to Inline gives you the possibility to write your own message.

The “Recipients” field of the mail reporter can take a list of recipients in any of the following syntaxes:

regular: john.doe@here.com, jane.smith@there.org
JSON-array style: ["john.doe@here.com", "jane.smith@there.org"]

To send one individualized mail to each recipient, the syntax for the “Recipients” field is :

{
   "mails": [
      {
         "to": "[email protected]",
         "variables": {
            "variable1": "Sir",
            "variable2": "some special message for john"
         }
      },
      {
         "to": "[email protected]",
         "variables": {
            "variable1": "Madam"
         }
      }
   ]
}

The variables object for each recipient is then available for replacement in the mail message as a JSON string mailVariables, and its fields as mailVariables_variableName. For example, you can type:

Dear ${mailVariables_variable1},

${if(parseJson(mailVariables).hasField("variable2"), parseJson(mailVariables).get("variable2"), "message for not-john")}

Yours truly,
    A humble documentation

Slack, Microsoft Teams, Webhook and Twilio reporters¶

These reporters only offer variable-based formatting using DSS formulas for the message body. These reporters also take additional parameters, like message color or sender alias, that can be computed using DSS formulas. Note that Slack uses a specific format for its messages, and leaves html as-is.

Slack integration setup¶

Slack provides 2 methods to automatically send messages on a channel, and both are available in DSS:

Through an incoming webhook
Through a bot user

To use an incoming webhook in a DSS integration, specify “Mode” as Use incoming webhook and provide a value for the “Webhook URL”. In Slack, you can find the webhook URL by navigating to Apps & Integrations -> Manage -> Custom integrations -> Incoming webhooks. To create an incoming webhook in your Slack channel, go to Apps & Integrations -> Build (top right corner) -> Make a Custom integration, and from there you can create a new Incoming webhook.

A bot user has the advantage (over a simple incoming webhook) that the bot can have a preset appearance in the Slack channel. Using a bot user in a DSS integration means selecting the API mode and specifying the bot’s API token as authentication token. The API token can be found by navigating to Apps & Integrations -> Build -> Make a Custom integration -> Bots.

Alternatively, you can use a testing token instead of a bot user, since they both rely on the same API token mechanism. Once they are created, you can access the token for your bot or incoming webhook by going to Apps & Integrations -> Manage (top right corner) -> Custom integrations.

Microsoft Teams integration setup¶

Integration with Microsoft Teams from DSS requires that you set up and configure an incoming webhook from within Microsoft Teams. Once configured, the integration with DSS supports simple JSON payloads, as well as more complex and rich cards created via JSON.

Once configured, the integration with DSS supports simple messages, as well as more complex and rich cards created via JSON. Here is an example of a card with displays a different banner and icon depending on the outcome of the scenario:

{
    "@type": "MessageCard",
    "@context": "https://schema.org/extensions",
    "themeColor": "${if(outcome == 'SUCCESS', '29AF5D', '')}${if(outcome == 'FAILED', 'F44336', '')}${if(outcome == '', '28A9DD', '')}",
    "summary": "${scenarioName} run report",
    "sections": [
        {
            "text": "${if(outcome == 'SUCCESS', '&#x2705;', '')}${if(outcome == 'FAILED', '&#x1F534;', '')}${if(outcome == '', '&#x1F514;', '')} [${scenarioName}](${scenarioRunURL}): **${outcome}**",
            "facts": [
                { "name": "Project", "value": "${scenarioProjectKey}" },
                { "name": "Triggered by", "value": "${triggerName}" }
            ]
        }
    ],
    "potentialAction": [
        {
            "@type": "OpenUri",
            "name": "View in DSS",
            "targets": [
                { "os": "default", "uri": "${scenarioRunURL}" }
            ]
        }
    ]
}

A key advantage of this integration is the ability to utilize the power of working on data projects in DSS while harnessing the ease of communication and collaboration that Microsoft Teams provides to users.

Note that you can also send messages to Teams in custom scenarios. See Custom scenarios for some examples.

Shell reporter¶

DSS sends results of the scenario run to the shell script. You can use DSS formulas in the Administration section to specify values for environment variables that define the shell script.

Scenario run URL¶

In order to use the reporter variable ${scenarioRunURL}, an administrator must configure the “DSS URL” under Administration > Settings > Notifications & Integrations > DSS Location > DSS URL.

Common variables¶

You can use variables from the DSS instance, project, and scenario run, in the message. Two mechanisms are also available to help with customizing the message:

Define new variables with a custom Python script.
Handle variables in the message as DSS formulas.