Intro to the Data Pipeline

Chartio’s Data Pipeline allows you to perform transformations on your query results in a series of steps before charting it. These steps include a variety of operations such as column sorting, pivoting data, and adding calculated columns. The flexibility of adding any steps in any order allows you to get your data exactly how you want it.

Data Pipeline view

Merging Data (Joins)

When adding two or more datasets in the Data Explorer, you will need to select a method for merging, or blending, the datasets. Merging datasets allows for powerful post-processing and calculations using Chartio’s drag and drop interface.

For every join option except Union and Cross Join, you can choose how many columns you would like to join the datasets on. View an example.

Merging layers in the Pipeline

Outer Join

Combines the columns from all datasets on one or more common dimension when possible, and includes all data from both datasets.

Outer join

Inner Join

Combines the columns on a common dimension (the first N columns) when possible, and only includes data for the columns that share the same values in the common N column(s).

Inner join

Left Join

Combines the columns on a common dimension (the first N columns) when possible, returning all rows from the first dataset with the matching rows in the second dataset. The result is NULL in the second dataset when there is no match.

Left join

Union

A Union merge will stack the dataset results on top of each other without grouping or combining the data. Unions can be used to generate lists of data to be printed or viewed in table format. To remove duplicate rows, check the Distinct checkbox.

Union

Cross Join

The result of the Cross Join will be a table with all possible combinations of your datasets together. This can result in enormous tables and should be used with caution. Cross Joins will likely only be used when your datasets are returning single values.

Cross Join

Adding a Step

Click the plus button in the pipeline wherever you want to apply a new transformation step to your query results. You can add as many steps as you’d like, in any order. You can also use each step multiple times as needed.

Preview Data

To preview your to view your data at any point in the pipeline, click the View Data link in a pipeline step. It can be incredibly useful to compare how your data looks before and after a transformation step is applied.

Data Pipeline Steps

For descriptions and examples of the available Data Pipeline steps, check out our Data Pipeline documentation.