So, you’ve been tasked with building an analytics dashboard. It’s tempting to jump into development straight away, but hold on a minute! There are numerous pitfalls that are easy to fall into and can ruin your plans for an attractive, useful dashboard. Here are five important principles for dashboard development to keep in mind every time you open up Power BI, Tableau, Looker, or any other BI tool.
1. Keep it focused and defined.
Before you start answering questions, you need to know exactly what you’re trying to find out. The starting point of most any dashboarding project should be a whiteboarding session with the end users; the dashboard becomes a collection of visuals that hold the ability to answer their questions.
For every single visual you create, make sure you’re answering a specific question. Each graph needs to be intentional and purposeful, and it’s very important to have your KPIs clearly defined well before you start building. If you don’t include your stakeholders from the very beginning, you’ll almost certainly have a lot more reworking to do after initial production is complete.
Courtesy of discourse.looker.com
2. A good data foundation is key.
Generating meaningful visualizations is nearly impossible without a good data foundation. Unclean data means holes and problems will need to be patched and fixed further down the pipeline. Many BI tools have functions that can format/prepare your data and generate some level of relational modeling for building your visualizations. However, too much modeling and logic in the tool itself will lead to large performance issues, and most BI tools aren’t specifically built with data wrangling in mind. A well-modeled semantic layer in a separate tool that handles all the necessary business logic is often essential for performance and governance.
Don’t undervalue the semantic layer!
The semantic layer is the step in preparation where the business logic is performed, joins are defined, and data is formatted from its raw form so it’s understandable and logical for users going forward. For Power BI users, for example, you would likely generate tabular models within SSAS. With a strong semantic layer in place before you even get to the BI tool, there will be little to no data management to be done in the tool itself. This means there is less processing the BI tool needs to handle and a much cleaner governance system.
In many BI tools, you can load in a raw dataset and have a functional dashboard in 10 minutes. However, building a semantic layer forces you to slow down and put some time in upfront for definition, development, and reflection about what the data is and what insights you’re trying to get for your business. This ensures you’re actually answering the right questions.
This is one of the many strengths of Looker, which is built specifically to handle the semantic layer as well as create visualizations. It forces you to define the logic in the tool itself before you start creating visuals.
It’s often tempting to skip the data prep steps in favor of putting out a finished product quickly, but remember: Your dashboard is only as good as the data underneath it.
3. PLEASE de-clutter.
There are numerous, obvious problems with the dashboard below, but there is one lesson to learn that many developers forget: Embrace white space! White space wants to be your friend. Like in web development, trying to pack too many visuals into the same dashboard is a recipe for disaster. Edward Tufte calls it the “data to ink ratio” in his book The Visual Display of Quantitative Information, one of the first and most impactful resources on data visualization.
Basically, just remove anything that isn’t essential or move important but non-pertinent information to a different page of the dashboard/report.
4. Think before using that overly complicated visual.
About to use a tree-map to demonstrate relationships among three variables at once? What about a 3-D, three-axis representation of sales? Most of the time: don’t. Visualizing data isn’t about making something flashy – it’s about creating something simple that someone can gain insight from at a glance. For almost any complex visualization, there is a simpler solution available, like splitting up the graph into multiple, more focused graphs.
5. Keep your interface clean, understandable, and consistent.
In addition to keeping your data clean and your logic well-defined, it’s important to make sure everything is understandable from start to finish and is easy to interpret by the end users. This starts with simply defining dimensions and measures logically and uniformly, as well as hiding excess and unused columns in the end product. A selection panel with 10 well-named column options is much easier than one with 30, especially if end-users will be doing alterations and exploration themselves.
You may notice a theme with most of these principles for dashboard development: Slow down and plan. It’s tempting to jump right into creating visuals, but never underestimate the value of planning and defining your steps first. Doing that will help ensure your dashboard is clean, consistent, and most important, valuable.
If you need help planning, implementing, or finding insights in your dashboards, the 2nd Watch team can help. Our certified consultants have the knowledge, training, and experience to help you drive the most value from your dashboard tool. Contact us today to learn about our data visualization starter pack.
Regardless of your industry or function, the ability to access, analyze, and make use of your data is essential. For many organizations, however, data is scattered throughout the organization in various applications (data silos), often in a format that’s unique to that system. The result is inconsistent access to data and unreliable insights. Some organizations may have a data management solution in place, such as a legacy or on-premise data warehouse, that is not able to keep up with the volume of data and processing speeds required for modern analytics tools or data science initiatives. For organizations striving to become data-driven, these limitations are a major roadblock.
The solution for many leading companies is a modern data warehouse
Over the course of several blogs, we tap into our extensive data warehouse experience across industry, function, and company sizes to guide you through this powerful data management solution.
Outline the different types of modern data warehouses.
Illustrate how the modern data warehouse fits in the big picture.
Share options on how to get started.
A modern data warehouse, implemented correctly, will allow your organization to unlock data-driven benefits from improved operations through data insights to machine learning to optimize sales pipelines. It will not only improve the way you access your data but will be instrumental in fueling innovation and driving business decisions in all facets of your organization.
Part 1: What Is a Data Warehouse?
At its most basic level, a data warehouse stores data from various applications and combines it together for analytical insights. The integrated data is then evaluated for quality issues, cleansed, organized, and modeled to represent the way a business uses the information – not the source system definition. With each business subject area integrated into the system, this data can be used for upstream applications, reporting, advanced analytics, and most importantly, for providing the insights necessary to make better, faster decisions.
Mini Case Study:
A great example of this is JD Edwards data integration. 2nd Watch worked with a client that had multiple source systems, including several JDE instances (both Xe and 9.1), Salesforce, TM1, custom data flows, and a variety of flat files they wanted to visualize in a dashboard report. The challenge was the source system definitions from JDE, with table names like “F1111”, Julian-style dates, and complex column mapping; it was nearly impossible to create the desired reports and visualizations.
2nd Watch solved this by creating a custom data architecture to reorganize the transactional data; centralize it; and structure the data for high-performance reporting, visualizations, and advanced analytics.
Image 1: The image above illustrates a retailer with multiple locations each with a different point of sale system. When they try to run a report on the numbers of units sold by state directly from data housed in these systems, the result is inaccurate due to data formatting inconsistencies. While this is a very simple example, imagine this on an enterprise scale.
Image 2: The image above shows the same data being run through an ETL process into a data warehouse. The result is a clear and accurate chart with the business users’ needs.
Data warehouses then . . . and now
There was a time when a data warehouse architecture consisted of a few source systems, a bunch of ELT/ETL (extract, transform, load) processes, and several databases, all running on one or two machines in an organization’s own data center. Companies would spend years building out this architecture with custom data processes that were used to copy and transform data from one database to another.
Times have changed and traditional on-premise data warehousing has hit its limits for most organizations. Enterprises have built data warehouse solutions in an era where they had limited data sources, infrequent changes, fewer transactions, and low competition. Now, the same systems that have been the backbone of an organization’s analytical environment are being rendered obsolete and ineffective.
Today’s organizations have to analyze data from many data sources to remain competitive. In addition, they are also addressing an increased volume of data coming from those data sources. Beyond this, in today’s fast-changing landscape, access to near real-time or instantaneous insights from data is necessary. Simply put, the legacy warehouse was not designed for the volume, velocity, and variety of data and analytics demanded by modern organizations.
If you depend on your data to better serve your customers, streamline your operations, and lead (or disrupt) your industry, a modern data warehouse built on the cloud is a must-have for your organization. In our next blog, we’ll dive deeper into the modern data warehouse and explore some of the options for deployment.
Contact us to learn what a modern data warehouse would look like for your organization.
Using a modern data warehouse, like Snowflake, can give your organization improved access to your data and dramatically improved analytics. When paired with a BI tool, like Tableau, or a data science platform, like Dataiku, you can gain even faster access to impactful insights that help your organization fuel innovation and drive business decisions.
In this post, we’ll provide a high-level overview of Snowflake, including a description of the tool, why you should use it, pros and cons, and complementary tools and technologies.
Overview of Snowflake
Snowflake was built from the ground up for the cloud, initially starting on AWS and scaling to Azure and GCP. With no servers to manage and near-unlimited scale in compute, Snowflake separates compute from storage and charges based on the size and length of time that compute clusters (known as “virtual warehouses”) are running queries.
Cross cloud lets organizations choose the cloud provider to use
Dynamic compute scaling saves on cost
Micro-partitioned storage with automatic maintenance
Rapid auto-scaling of compute nodes allows for increased cost savings and high concurrency on demand, and compute and storage are separated
Built for MPP (massive parallel processing)
Optimized for read via a columnar backend
Dedicated compute means no concurrency issues
Ability to assign dedicated compute
High visibility into spend
Native support for JSON, XML, Avro, Parquet, and ORC semi-structured data formats
SnowSQL has slight syntax differences
Introduction of Snowpark for Snowflake native development
Full visibility into queries executed, by whom, and how long they ran
Precision point-in-time restore available via “time-travel” feature
Why Use Snowflake
Decoupled from cloud vendors, it allows a true multi-cloud experience. You can deploy on Azure, AWS, GCP, or any combination of those cloud services. With near-unlimited scale and minimal management, it offers a best-in-class data platform but with a pay-for-what-you-use consumption model.
Pros of Snowflake
Allows for a multi-cloud experience built on top of existing AWS, Azure, or GCP resources, depending on your preferred platform
Easy implementation of security and role definitions for less frustrating user experience and easier delineation of cost while keeping data secure
Integrated ability to share data to partners or other consumers outside of an organization and supplement data with publicly available datasets within Snowflake
Cons of Snowflake
Ecosystem of tooling continues to grow as adoption expands, but some features are not readily available
Due to the paradigm shift in a cloud-born architecture, taking full advantage of Snowflake’s advanced features requires a good understanding of cloud data architecture
Select Complementary Tools and Technologies for Snowflake
Azure Data Factory
We hope you found this high-level overview of Snowflake helpful. If you’re interested in learning more about Snowflake or other modern data warehouse tools like Amazon Redshift, Azure Synapse, and Google BigQuery, contact us to learn more.