Writing DSS macros

For more information about macros, see DSS Macros.

To start creating a macro, we recommend that you use the plugin developer tools (see the tutorial for an introduction). In the Definition tab, click on “Add Python macro”, and enter the identifier for your new macro. You’ll see a new folder python-runnables and will have to edit the runnable.py and runnable.json files

A macro is essentially a Python function, wrapped in a class, written in a runnable.py file in the macro’s folder.

A basic macro’s code looks like

from dataiku.runnables import Runnable

class MyMacro(Runnable):
    def __init__(self, project_key, config, plugin_config):
        self.project_key = project_key
        self.config = config

    def get_progress_target(self):
        return None

    def run(self, progress_callback):
        # Do some things. You can use the dataiku package here

        result = "It worked"
        return result

The associated runnable.json file looks like:

{
    "meta" : {
        "label" : "A great macros",
        "description" : "It does stuff",
        "icon" : "icon-trash"
    },

    "impersonate" : false,
    "permissions" : ["READ_CONF"],
    "resultType" : "HTML",
    "resultLabel" : "THe output",

    "params": [
        {
            "name": "param_name",
            "label" : "The parameter",
            "type": "INT",
            "description":"Delete logs for jobs older than this",
            "mandatory" : true,
            "defaultValue" : 15
        }
    ]
}

The “meta” and “params” fields are similar to all other kinds of DSS components.

Result of a macro

In addition to performing its action, a macro can return a result, which will be displayed by the user. In many cases, the main job of a macro is to output some kind of report. In that case, the result is actually the main function of the macro.

To return a result from your macro, you must first define the resultType field in the runnable.json file.

Valid result types are defined below

HTML

In runnable.json, set "resultType" : "HTML"

Your macro’s run function must return a HTML string, which will be displayed inline in the result’s page. Users will have the option to download the HTML. You may use CSS declarations in your HTML code but please make sure to properly scope them so that they cannot interfere with DSS.

URL

In runnable.json, set "resultType" : "URL"

Your macro’s run function must return an URL as a string. Users will be presented with a link.

RESULT_TABLE

In runnable.json, set "resultType" : "RESULT_TABLE".

This allows you to build a table view which will be properly formatted for display. We recommend that you use RESULT_TABLE rather than HTML if the output of your macro is a simple table, as you won’t have to handle styling and formatting.

In your macro’s run function, create and fill your result table as follows

from dataiku.runnables import Runnable, ResultTable

rt = ResultTable()

# First, declare the columns of the output
# Parameters to add_Column are: id, label, type
rt.add_column("dataset", "Dataset", "STRING")

rt.add_column("table", "Table", "STRING")

# Then, add records, as lists, in the same order as the columns
record = []
record.append("dataset_name")
record.append("table_name")
rt.add_record(record)

# Return the result table as the return value of the macro's run function
return rt

Valid types for columns are:

  • STRING: A regular string
  • STRING_LIST: Add a list of strings in the record array. It will be formatted comma-separated

Interacting with DSS in macros

The recommended way to interact with DSS in the code of a macro is either the internal Python API or the public API.

For internal API, for example, this includes dataiku.Dataset(). Interaction with the public API is made easy by dataiku.api_client() which gives you a public API client handle, automatically configured with the permissions of the user running the macro.

Progress reporting

To be written

Security of macros

Impersonation

You need to configure whether this macro will run with UNIX identity of the user running DSS, or with the identity of the final user. Note that this is only relevant for the users of your plugin if Multi-user security is enabled, but since your plugin might be used in both cases, you still need to take care of this.

Generally speaking, we recommend that your macros run with "impersonate" : true. This means that they may not access the filesystem outside of their working directory and should only use DSS APIs for their operations.

If your macro runs with "impersonate": false, it can access the filesystem, notably the DSS datadir.

Permissions

A macro always runs in the context of a project (which is passed to the macro’s constructor).

Your macro can define the project permissions that users must have to be able to run the macro. This is done in the permissions field of the runnable.json file.

Valid permission identifiers are:

  • ADMIN
  • READ_CONF
  • WRITE_CONF

For more information about the meaning of these permissions, see Main permissions

In addition, if the users running the macro need to have global DSS administrator rights, set the "requiresGlobalAdmin" field of runnable.json to true