Scenarios¶

The scenario API can control all aspects of managing a scenario.

Note

There is a dedicated API to use within a scenario to run steps and report on progress of the scenario. For that, please see Scenarios (in a scenario).

Example use cases
Reference documentation

Example use cases ¶

In all examples, project is a dataikuapi.dss.project.DSSProject handle, obtained using client.get_project() or client.get_default_project()

Run a scenario ¶

Variant 1: Run and wait for it to complete ¶

scenario = project.get_scenario("myscenario")

scenario.run_and_wait()

Variant 2: Run, then poll while doing other stuff ¶

scenario = project.get_scenario("myscenario")

trigger_fire = scenario.run()

# When you call `run` a scenario, the scenario is not immediately
# started. Instead a "manual trigger" fires.
#
# This trigger fire can be cancelled if the scenario was already running,
# or if another trigger fires
# Thus, the scenario run is not available immediately, and we must "wait"
# for it

scenario_run = trigger_fire.wait_for_scenario_run()

# Now the scenario is running. We can wait for it synchronously with
# scenario_run.wait_for_completion(), but if we want to do other stuff
# at the same time, we can use refresh

while True:
    # Do a bit of other stuff
    # ...

    scenario_run.refresh()
    if scenario_run.running:
        print("Scenario is still running ...")
    else:
        print("Scenario is not running anymore")
        break

    time.sleep(5)

Get information about the last completed run of a scenario ¶

scenario = project.get_scenario("myscenario")

last_runs = scenario.get_last_runs(only_finished_runs=True)

if len(last_runs) == 0:
    raise Exception("The scenario never ran")

last_run = last_runs[0]

# outcome can be one of SUCCESS, WARNING, FAILED or ABORTED
print("The last run finished with %s" % last_run.outcome)

# start_time and end_time are datetime.datetime objects
print("Last run started at %s and finished at %s" % (last_run.start_time, last_run.end_time))

Disable/enable scenarios ¶

Disable and remember ¶

This snippet disables all scenarios in a project (i.e. prevents them from auto-triggering), and also keeps a list of the ones that were active, so that you can selectively re-enable them later

# List of scenario ids that were active
previously_active = []

for scenario in project.list_scenarios(as_type="objects"):
    settings = scenario.get_settings()

    if settings.active:
        previously_active.append(scenario.id)
        settings.active = False
        # In order for settings change to take effect, you need to save them
        settings.save()

Enable scenarios from a list of ids ¶

for scenario_id in previously_active:
    scenario = project.get_scenario(scenario_id)
    settings = scenario.get_settings()

    settings.active = True
    settings.save()

List the “run as” user for all scenarios ¶

This snippet allows you to list the identity under which a scenario runs:

for scenario in project.list_scenarios(as_type="objects"):
    settings = scenario.get_settings()
    # We must use `effective_run_as` and not `run_as` here.
    # run_as contains the "configured" run as, which can be None - in that case, it will run
    # as the last modifier of the scenario
    # effective_run_as is always valued and is the resolved version.
    print("Scenario %s runs as user %s" % (scenario.id, settings.effective_run_as))

Reassign scenarios to another user ¶

If user “u1” has left the company, you may want to reassign all scenarios that ran under his identity to another user “u2”.

for scenario in project.list_scenarios(as_type="objects"):
    settings = scenario.get_settings()

    if settings.effective_run_as == "u1":
        print("Scenario %s used to run as u1, reassigning it")
        # To configure a run_as, we must use the run_as property.
        # effective_run_as is read-only
        settings.run_as = "u2"
        settings.save()

Get the “next expected” run for a scenario ¶

If the scenario has a temporal trigger enabled, this will return a datetime of the approximate next expected run

scenario = project.get_scenario("myscenario")
# next_run is None if no next run is scheduled
print("Next run is at %s" % scenario.get_status().next_run)

Get the list of jobs started by a scenario ¶

“Build/Train” or Python steps in a scenario can start jobs. This snippet will give you the list of job ids that a particular scenario run executed.

These job ids can then be used together with dataikuapi.dss.project.DSSProject.get_job()

scenario = project.get_scenario("myscenario")
# Focusing only on the last completed run. Else, use get_last_runs() and iterate
last_run = scenario.get_last_finished_run()

last_run_details = last_run.get_details()

all_job_ids = []
for step in last_run_details.steps:
    all_job_ids.extend(step.job_ids)

print("All job ids started by scenario run %s : %s" % (last_run.id, all_job_ids))

Get the first error that happened in a scenario run ¶

This snippet retrieves the first error that happened during a scenario run.

scenario = project.get_scenario("myscenario")

last_run = scenario.get_last_finished_run()

if last_run.outcome == "FAILED":
    last_run_details = last_run.get_details()
    print("Error was: %s" % (last_run_details.first_error_details))

Start multiple scenarios and wait for all of them to complete ¶

This code snippet starts multiple scenarios and returns when all of them have completed, returning the updated DSSScenarioRun for each

import time

scenarios_ids_to_run = ["s1", "s2", "s3"]

scenario_runs = []

for scenario_id in scenarios_ids_to_run:
    scenario = project.get_scenario(scenario_id)

    trigger_fire = scenario.run()
    # Wait for the trigger fire to have actually started a scenario
    scenario_run = trigger_fire.wait_for_scenario_run()
    scenario_runs.append(scenario_run)

# Poll all scenario runs, until all of them have completed
while True:
    any_not_complete = False
    for scenario_run in scenario_runs:
        # Update the status from the DSS API
        scenario_run.refresh()
        if scenario_run.running:
            any_not_complete = True

    if any_not_complete:
        print("At least a scenario is still running...")
    else:
        print("All scenarios are complete")
        break

    # Wait a bit before checking again
    time.sleep(30)

print("Scenario run ids and outcomes: %s" % ([(sr.id, sr.outcome) for sr in scenario_runs]))

Change the “from” email for email reporters ¶

Note that usually, we would recommend using variables for “from” and “to” email. But you can also modify them with the API.

scenario = project.get_scenario("myscenario")

settings = scenario.get_settings()

for reporter in settings.raw_reporters:
    # Only look into 'email' kind of reporters
    if reporter["messaging"]["type"] == "mail-scenario":
        messaging_configuration = reporter["messaging"]["configuration"]
        messaging_configuration["sender"] = "[email protected]"
        print("Updated reporter %s" % reporter["id"])

settings.save()

Reference documentation ¶

class dataikuapi.dss.scenario.DSSScenario(client, project_key, id)¶

A handle to interact with a scenario on the DSS instance. Do not create this directly, use dataikuapi.dss.DSSProject.get_scenario()

abort()¶: Aborts the scenario if it currently running. Does nothing if the scenario is not currently running

run_and_wait(params=None, no_fail=False)¶

Requests a run of the scenario, which will start after a few seconds. Wait the end of the run to complete.

Parameters: params (dict) – additional parameters that will be passed to the scenario through trigger params (defaults to {})
Returns: A dataikuapi.dss.admin.DSSScenarioRun run handle

run(params=None)¶

Requests a run of the scenario, which will start after a few seconds.

Params dict params: additional parameters that will be passed to the scenario through trigger params (defaults to {})

:return the trigger fire object. Note that this is NOT a Scenario run object. The trigger fire may ultimately result in a run or not. :rtype DSSTriggerFire

get_last_runs(limit=10, only_finished_runs=False)¶

Get the list of the last runs of the scenario

Returns: A list of dataikuapi.dss.scenario.DSSScenarioRun

get_runs_by_date(from_date, to_date=datetime.datetime(2023, 6, 25, 9, 15, 43, 426436))¶

Get the list of the runs of the scenario in a given date range, [from_date, to_date)

Parameters

from_date (datetime) – start of the date range to retrieve runs for, inclusive
to_date (datetime) – end of the date range to retrieve runs for, or now(), exclusive

Returns

A list of dataikuapi.dss.scenario.DSSScenarioRun

get_last_finished_run()¶: Gets the last run that completed (successfully or not) :return: A dataikuapi.dss.scenario.DSSScenarioRun

get_last_successful_run()¶: Gets the last run that completed successfully :return: A dataikuapi.dss.scenario.DSSScenarioRun

get_current_run()¶

Get the current run of the scenario, or None if it is not running at the moment

Returns: A dataikuapi.dss.scenario.DSSScenarioRun

get_run(run_id)¶

Get a handle to a run of the scenario based on a scenario run id

Returns: A dataikuapi.dss.scenario.DSSScenarioRun

get_status()¶

Returns the status of this scenario

Return type: DSSScenarioStatus

get_settings()¶

Returns the settings of this scenario

Return type: StepBasedScenarioSettings or PythonScriptBasedScenarioSettings

get_average_duration(limit=3)¶

Get the average duration (in fractional seconds) of the last runs of this scenario that finished, where finished means it ended with SUCCESS or WARNING. If there are not enough runs to perform the average, returns None

Args:: limit: number of last runs to average on

delete()¶: Deletes this scenario

get_object_discussions()¶

Get a handle to manage discussions on the scenario

Returns: the handle to manage discussions
Return type: dataikuapi.discussion.DSSObjectDiscussions

get_trigger_fire(trigger_id, trigger_run_id)¶: Gets the trigger fire object corresponding to a previous trigger fire id. Advanced usages only

get_definition(with_status=True)¶

Deprecated, use get_settings() and get_summary()

Returns the definition of the scenario :param bool status: if True, the definition contains the run status of the scenario but not its

actions’ definition. If False, the definition doesn’t contain the run status but has the scenario’s actions definition

set_definition(definition, with_status=True)¶

Deprecated, use get_settings() and DSSScenarioSettings.save()

Updates the definition of this scenario :param bool status: should be the same as the value passed to get_definition(). If True, the params,

triggers and reporters fields of the scenario are ignored,

get_payload(extension='py')¶

Deprecated, use get_settings() and DSSScenarioSettings.save()

Returns the payload of the scenario :param str extension: the type of script. Default is ‘py’ for python

set_payload(script, with_status=True)¶

Deprecated, use get_settings() and DSSScenarioSettings.save()

Updates the payload of this scenario :param str extension: the type of script. Default is ‘py’ for python

class dataikuapi.dss.scenario.DSSScenarioStatus(scenario, data)¶

Status of a scenario. Do not instantiate this class, use DSSScenario.get_status()

get_raw()¶

property running¶

property next_run¶

Time at which the scenario is expected to next run based on its triggers.

May be None if there are no temporal triggers

This is an approximate indication as scenario run may be delayed, especially in the case of multiple triggers or high load

class dataikuapi.dss.scenario.DSSScenarioSettings(client, scenario, data)¶

Settings of a scenario. Do not instantiate this class, use DSSScenario.get_settings()

get_raw()¶

property active¶: Whether this scenario is currently active, i.e. its auto-triggers are executing

property run_as¶: The configured ‘run as’ of the scenario. None means that the scenario runs as its last modifier. Only administrators may set a non-None value

property effective_run_as¶

The effective ‘run as’ of the scenario. If a run_as has been configured by an administrator, this will be used. Else, this will be the last modifier of the scenario.

If this method returns None, it means that it was not possible to identify who this scenario runs as. This scenario is probably not currently functioning.

property raw_triggers¶

property raw_reporters¶

add_periodic_trigger(every_minutes=5)¶: Adds a trigger that runs the scenario every X minutes

add_hourly_trigger(minute_of_hour=0, year=None, month=None, day=None, starting_hour=0, repeat_every=1, timezone='SERVER')¶: Adds a trigger that runs the scenario each hour on a predefined minute

add_daily_trigger(hour=2, minute=0, days=None, year=None, month=None, day=None, repeat_every=1, timezone='SERVER')¶

Adds a trigger that runs the scenario each day on a predefined time.

Parameters: list (day) – if not None, only runs on given days. Day must be given as english names with capitals

add_monthly_trigger(day=1, hour=2, minute=0, year=None, month=None, run_on='ON_THE_DAY', repeat_every=1, timezone='SERVER')¶: Adds a trigger that runs the scenario once per month

save()¶: Saves the settings to the scenario

class dataikuapi.dss.scenario.StepBasedScenarioSettings(client, scenario, data)¶

property raw_steps¶: Returns raw definition of steps

class dataikuapi.dss.scenario.PythonScriptBasedScenarioSettings(client, scenario, data, script)¶

property code¶

save()¶: Saves the settings to the scenario

class dataikuapi.dss.scenario.DSSScenarioRun(client, run)¶

A handle containing basic info about a past run of a scenario.

This handle can also be used to fetch additional information about the urn

property id¶: The run id of this run

refresh()¶: Refreshes the details (outcome, running, info, …) from the scenario

wait_for_completion(no_fail=False)¶: If the scenario run is not complete, wait for it to complete :param boolean no_fail: if no_fail=False, raises an exception if scenario fails

property running¶: Returns whether this run is running

property outcome¶: The outcome of this scenario run, if available :return one of SUCCESS, WARNING, FAILED, or ABORTED :rtype str

property trigger¶: The raw details of the trigger that triggered this scenario run :rtype dict

get_info()¶: Get the raw basic information of the scenario run :rtype dict

get_details()¶

Get the full details of the scenario run, including its step runs. Note: this performs another API call

:rtype dict

get_start_time()¶: Get the start time of the scenario run

property start_time¶: Get the start time of the scenario run

get_end_time()¶: Get the end time of the scenario run, if it completed. Else raises

property end_time¶: Get the end time of the scenario run, if it completed. Else raises

get_duration()¶

Get the duration of this run (in fractional seconds).

If the run is still running, get the duration since it started

property duration¶

Get the duration of this run (in fractional seconds).

If the run is still running, get the duration since it started

class dataikuapi.dss.scenario.DSSScenarioRunDetails(data)¶

property steps¶

property last_step¶

property first_error_details¶: Try to get the details of the first error if this run failed. This will not always be able to find the error details (it returns None in that case)

class dataikuapi.dss.scenario.DSSStepRunDetails(data)¶

property outcome¶

property job_ids¶: The list of DSS job ids that were ran as part of this step

property first_error_details¶: Try to get the details of the first error if this step failed. This will not always be able to find the error details (it returns None in that case)

class dataikuapi.dss.scenario.DSSScenarioRunWaiter(scenario_run, trigger_fire)¶

Helper to wait for a scenario to run to complete

wait(no_fail=False)¶

class dataikuapi.dss.scenario.DSSTriggerFire(scenario, trigger_fire)¶

A handle representing the firing of a trigger on a scenario. Do not create this class directly, use DSSScenario.run()

wait_for_scenario_run(no_fail=False)¶

Polls, waiting for the run of the sceanrio that this trigger activation launched to be available, or for the trigger fire to be cancelled (possibly cancelled by another firing)

Parameters: no_fail – If no_fail=True, will return None if the trigger fire is cancelled, else will raise

:returns a scenario run object, or None :rtype DSSScenarioRun

get_scenario_run()¶: Get the run of the scenario that this trigger activation launched. May return None if the scenario run started from this trigger has not yet been created

is_cancelled(refresh=False)¶

Whether the trigger fire has been cancelled

Parameters: refresh – get the state of the trigger from the backend

class dataikuapi.dss.scenario.DSSScenarioListItem(client, data)¶

An item in a list of scenarios. Do not instantiate this class, use :meth:`dataikuapi.dss.project.DSSProject.list_scenarios

to_scenario()¶: Gets the DSSScenario corresponding to this scenario

property id¶