Data changes all the time. Updating models, adding new fields due to new features or tooling, clarifying column names, cleaning unnecessary columns and all kinds of other data actions result in schemas changing.
Changing those schemas can be hugely powerful and make future data work easier and more democratized. It can however be an enormous pain when for instance simply renaming a column can impact dozens of charts! We’ve heard from our customers that this pain of migrating charts due to data changes keeps them from making those needed improvements. They stick with the mess longer, and avoid doing the incremental cleaning and modeling work needed for great data governance and a clear source of truth.
Today, we’re releasing a giant upgrade with key improvements in fixing these migration annoyances that we hope will further enable and encourage companies to take an agile incremental approach to data governance and modeling.
If you automate the building of your schemas with tools like dbt, Airflow, or other automation tools, you can coordinate your Chartio schema syncs using our API. We have added an additional API endpoint that triggers the synchronization process in order to keep Chartio up-to-date with the latest schema changes.
Non-breaking schema Syncs vs breaking Refreshed Schemas
Previously to reflect schema changes, you had to “Refresh” the schema and be prompted with an intimidating overview of what changes would be reflected. These changes often involved charts being switched to SQL mode if a column or table they were dependent on no longer existed or was simply renamed. Even for SQL based charts, it was difficult to find all of the charts that were erroring. We found it delayed people from performing their syncs and hugely upset many that didn’t quite realize the impacts that would be had by syncing their data source. It was overly strict and needed change.
Now, we never change a chart to SQL mode from Visual mode. A chart that depends on a column that no longer exists will clearly show you the issue:
On that chart, you can simply choose a different column.
We have greatly improved our “Sync Schema” (previously called “Refresh Schema”) functionality. Now, when you synchronize your schema between your database and Chartio, we automatically apply all updates. This “always in-sync” capability removes the time-consuming and unnecessary step of choosing which changes to synchronize. Now, Chartio will automatically sync all changes.
We’ve also improved our Schema tab to show which columns are referenced in Chartio, but no longer exist in the source database. This allows administrators to remap these columns quickly, in one step, instead of having to manually update every query.
Clear Change History
We added a tab that clearly shows the changes to schema that have occurred over time. This page logs all of the changes, tables affected, and has a nice change summary. This is designed to help administrators and users track down any specific changes that may have occurred.
Query and Error Search
We’ve added the ability to search for usage of tables, columns, or any snippet! Now, administrators can proactively search for usage of columns within Chartio before making any breaking changes in their database! Or this new search capability can be used to find broken queries within Chartio and fix them quickly without having to click through every dashboard, alert, or data store.
We also provide, as part of the search, a lot of useful information such as Last Run, Status, Owner, and a link to the query. You can also download the data as a CSV.
All of these upgrades are available now to all Chartio customers! We hope that making updates to your databases and schemas are much smoother going forward. Give them a try and send us your feedback, we’d love to hear from you. If you’d like a live deep dive and demo of these new features, please join us on December 15th at 11am PST.
