Manage a dataflow

A dataflow is the unit of processing a dataset. A single dataflow can be associated with multiple datasets to perform transformations. That is, a dataset must belong to a dataflow for transformation rules to be applied. It forms a relationship such as a “join” or “union” with other datasets.

As shown below, the dataflow details page shows the dependency among all datasets in a dataflow, and the transformation rules applied to each dataset.

The following subsections cover the processes involved in defining a dataflow, such as adding a dataset, editing transformation rules, and creating a data snapshot with transformation results.

The Dataflow menu can be accessed under MANAGEMENT > Data Preparation > Dataflow on the left-hand panel of the main screen.

• Add a dataset
  • Adding a dataset after creating an empty dataflow
  • Creating a dataflow in the dataset details page
• Edit rules
  • Create a rule
    • Using the column header menu
    • Using the command input panel
    • Inserting into a rule list
  • Edit a created rule
    • Editing a rule
    • Deleting a rule
    • Undo and redo
• Rule types
  • drop
  • header
  • settype
  • setformat
  • rename
  • keep
  • delete
  • replace
  • set
  • derive
  • split
  • merge
  • extract
  • countpattern
  • nest
  • unnest
  • flatten
  • aggregate
  • pivot
  • unpivot
  • join
  • union
  • window
• Create a data snapshot