The Flow

In DSS, the datasets and the recipes together make up the flow. We have created a visual grammar for data science, so users can quickly understand a data pipeline through the flow.

Using the flow, DSS knows the lineage of every dataset in the flow. DSS, therefore, is able to dynamically rebuild datasets whenever one of their parent datasets or recipes has been modified.

DSS can limit the resource usage of jobs building datasets. Users can configure their recipes and administrators can configure DSS to prevent some users, projects, recipes or plugins from consuming too many resources.

• Visual Grammar
  • Datasets
  • Visual Recipes
  • Machine Learning
  • Code Recipes
  • Plugin Recipes
• Flow zones
  • Use cases
  • Usage
• Rebuilding Datasets
  • Build
  • Build Flow outputs reachable from here
  • Propagate schema across Flow from here
• Limiting Concurrent Executions
  • Global and per-job limits
  • Per-recipe limits
  • Additional limits
• Exporting the Flow to PDF or images
  • Setup
  • Manual usage
  • Settings
• How to Manage Large Flows with Flow Folding
  • Folding and Unfolding a Flow
• Flow Document Generator
  • Generate a flow document
  • Custom templates