Six Questions to Save Time and Money on Data Warehousing

Your quest for reliable information can be sidetracked by decisions about information technology.

Even after you’ve decided that you need a business intelligence system, it’s easy to get lost in questions about data storage. How is my data stored, and how should it be? What kind of data storage does my business intelligence tool need? What’s a data warehouse? Do I need one? If so, what kind of data warehouse do I need?

It can be confusing to know where to start.

Chartio’s new white paper Read this before you make a data warehouse decision: Data warehousing for non-IT Executives is designed to help non-IT executives understand the concepts behind data warehousing, so they can work with IT to determine their needs.

A data warehouse can cost a lot more than your business intelligence tool itself, require additional development resources, impose long term maintenance requirements, complicate your architecture, and delay implementation of your BI project. If you can avoid using a data warehouse, you probably should.

If your data is already stored in a SQL database, you may not need a data warehouse. Many of our customers either clone their operational database, or maintain a simplified summary of their live data that contains only the information they need for analytical and BI queries. This takes the load off their operational database without creating the need for additional hardware and software.

If your data is already stored on Amazon Redshift, you may not need a separate data warehouse for business intelligence. Amazon Redshift is designed to be used as a data warehouse, and many business intelligence tools can connect directly to it.

What Kinds of Data Require Warehousing?

However, there are some good reasons to build a data warehouse. Your answers to the following questions will help you understand whether you need a data warehouse, and what kind of data warehouse you need.

• How big is your data? If you have huge volumes of data (say, more than 100 million rows), you may need to move your data to a system that can support large scans and fast aggregations. Fortunately, today there are cloud solutions to simplify this process.

• How messy is your data? If your data needs to be aggregated, anonymized, or cleaned in order to be quickly and usefully analyzed, it may make sense to periodically preprocess the data and store it separately. If your dataset isn’t huge, and your data isn’t messy, you may be able to either directly query your data or an exact copy.

• Can you use aggregated data? It’s possible to create dashboards that show the up-to-the-minute status of your business by analyzing a near-real-time clone of your operational database. But real-time data can take up a great deal of storage. If you’re more concerned with trends and daily/weekly/monthly/quarterly numbers, it may be preferable to create summary tables of your live data. There may also be other cases where you’ll want to aggregate detailed data to save storage or secure customer information.

• Where and how is your data stored? if your data is stored in a SQL database, such as MySQL or PostgreSQL, your BI tool may be able to access it directly, or it will be simple to maintain a copy for your BI tool to use. However, if some of your data is in sources with which your BI tool can’t connect, you may need to copy that data into a SQL database or data warehouse.

• Do you need to merge databases? If you need to merge data from multiple databases or disparate data sources (such as customer data from your website and your internal operational systems), you may need to use an ETL (Extract, Transform, and Load) process to build an analytical database. For more information about ETL, please download our white paper.

We created a diagram to help you navigate this decision process.

Decision tree for choosing a data architecture: it all comes down to time and money.

If you’ve found you don’t need a data warehouse, congratulations. You should focus on business intelligence systems that don’t require a data warehouse, such as Chartio.

If you’re using raw data, Chartio can connect directly with your database, or a frequently-updated clone.

If you need to add summary tables to either clean or aggregate your data, Chartio has a solution for you. Request a demo for more information.

If you think you may need a data warehouse, download our white paper to find out more about your options.

If you’re still not sure whether you need a data warehouse, download our white paper to learn more about how Agile Business Intelligence can simplify your decision making. Or request a demo, and we’ll help you determine your needs and make some suggestions about solutions.