The Flow¶
In Dataiku Agent Management, the datasets and the recipes together make up the flow. We have created a visual grammar for data science, so users can quickly understand a data pipeline through the flow.
Using the flow, Dataiku Agent Management knows the lineage of every dataset in the flow. Dataiku Agent Management, therefore, is able to dynamically rebuild datasets whenever one of their parent datasets or recipes has been modified.
Dataiku Agent Management can limit the resource usage of jobs building datasets. Users can configure their recipes and administrators can configure Dataiku Agent Management to prevent some users, projects, recipes or plugins from consuming too many resources.