Category: Data Strategy

Data & AI Predictions in 2023

As we reveal our data and AI predictions for 2023, join us at 2nd Watch to stay ahead of the curve and propel your business towards innovation and success. How do we know that artificial intelligence (AI) and large language models (LLMs) have reached a tipping point? It was the hot topic at most families’ dinner tables during the 2022 holiday break.

AI has become mainstream and accessible. Most notably, OpenAI’s ChatGPT took the internet by storm, so much so that even our parents (and grandparents!) are talking about it. Since AI is here to stay beyond the Christmas Eve dinner discussion, we put together a list of 2023 predictions we expect to see regarding AI and data.

#1. Proactively handling data privacy regulations will become a top priority.

Regulatory changes can have a significant impact on how organizations handle data privacy: businesses must adapt to new policies to ensure their data is secure. Modifications to regulatory policies require governance and compliance teams to understand data within their company and the ways in which it is being accessed. 

To stay ahead of regulatory changes, organizations will need to prioritize their data governance strategies. This will mitigate the risks surrounding data privacy and potential regulations. As a part of their data governance strategy, data privacy and compliance teams must increase their usage of privacy, security, and compliance analytics to proactively understand how data is being accessed within the company and how it’s being classified. 

#2. AI and LLMs will require organizations to consider their AI strategy.

The rise of AI and LLM technologies will require businesses to adopt a broad AI strategy. AI and LLMs will open opportunities in automation, efficiency, and knowledge distillation. But, as the saying goes, “With great power comes great responsibility.” 

There is disruption and risk that comes with implementing AI and LLMs, and organizations must respond with a people- and process-oriented AI strategy. As more AI tools and start-ups crop up, companies should consider how to thoughtfully approach the disruptions that will be felt in almost every industry. Rather than being reactive to new and foreign territory, businesses should aim to educate, create guidelines, and identify ways to leverage the technology. 

Moreover, without a well-thought-out AI roadmap, enterprises will find themselves technologically plateauing, teams unable to adapt to a new landscape, and lacking a return on investment: they won’t be able to scale or support the initiatives that they put in place. Poor road mapping will lead to siloed and fragmented projects that don’t contribute to a cohesive AI ecosystem.

#3. AI technologies, like Document AI (or information extraction), will be crucial to tap into unstructured data.

According to IDC, 80% of the world’s data will be unstructured by 2025, and 90% of this unstructured data is never analyzed. Integrating unstructured and structured data opens up new use cases for organizational insights and knowledge mining.

Massive amounts of unstructured data – such as Word and PDF documents – have historically been a largely untapped data source for data warehouses and downstream analytics. New deep learning technologies, like Document AI, have addressed this issue and are more widely accessible. Document AI can extract previously unused data from PDF and Word documents, ranging from insurance policies to legal contracts to clinical research to financial statements. Additionally, vision and audio AI unlocks real-time video transcription insights and search, image classification, and call center insights.

Organizations can unlock brand-new use cases by integrating with existing data warehouses. Finetuning these models on domain data enables general-purpose models across a wide variety of use cases. 

#4. “Data is the new oil.” Data will become the fuel for turning general-purpose AI models into domain-specific, task-specific engines for automation, information extraction, and information generation.

Snorkel AI coined the term “data-centric AI,” which is an accurate paradigm to describe our current AI lifecycle. The last time AI received this much hype, the focus was on building new models. Now, very few businesses need to develop novel models and algorithms. What will set their AI technologies apart is the data strategy.

Data-centric AI enables us to leverage existing models that have already been calibrated to an organization’s data. Applying an enterprise’s data to this new paradigm will accelerate a company’s time to market, especially those who have modernized their data and analytics platforms and data warehouses. 

#5. The popularity of data-driven apps will increase.

Snowflake recently acquired Streamlit, which makes application development more accessible to data engineers. Additionally, Snowflake introduced Unistore and hybrid tables (OLTP) to allow data science and app teams to work together and jointly off of a single source of truth in Snowflake, eliminating silos and data replication.

Snowflake’s big moves demonstrate that companies are looking to fill gaps that traditional business intelligence (BI) tools leave behind. With tools like Streamlit, teams can harness tools to automate data sharing and deployment, which is traditionally manual and Excel-driven. Most importantly, Streamlit can become the conduit that allows business users to work directly with the AI-native and data-driven applications across the enterprise.

#6. AI-native and cloud-native applications will win.

Customers will start expecting AI capabilities to be embedded into cloud-native applications. Harnessing domain-specific data, companies should prioritize building upon module data-driven application blocks with AI and machine learning. AI-native applications will win over AI-retrofitted applications. 

When applications are custom-built for AI, analytics, and data, they are more accessible to data and AI teams, enabling business users to interact with models and data warehouses in a new way. Teams can begin classifying and labeling data in a centralized, data-driven way, rather than manually and often-repeated in Excel, and can feed into a human-in-the-loop system for review and to improve the overall accuracy and quality of models. Traditional BI tools like dashboards, on the other hand, often limit business users to consume and view data in a “what happened?” manner, rather than in a more interactive, often more targeted manner.

#7. There will be technology disruption and market consolidation.

The AI race has begun. Microsoft’s strategic partnership with OpenAI and integration into “everything,” Google’s introduction of Bard and funding into foundational model startup Anthropic, AWS with their own native models and partnership with Stability AI, and new AI-related startups are just a few of the major signals that the market is changing. The emerging AI technologies are driving market consolidation: smaller companies are being acquired by incumbent companies to take advantage of the developing technologies. 

Mergers and acquisitions are key growth drivers, with larger enterprises leveraging their existing resources to acquire smaller, nimbler players to expand their reach in the market. This emphasizes the importance of data, AI, and application strategy. Organizations must stay agile and quickly consolidate data across new portfolios of companies. 

Conclusion

The AI ball is rolling. At this point, you’ve probably dabbled with AI or engaged in high-level conversations about its implications. The next step in the AI adoption process is to actually integrate AI into your work and understand the changes (and challenges) it will bring. We hope that our data and AI predictions for 2023 prime you for the ways it can have an impact on your processes and people.

Think you’re ready to get started? Find out with 2nd Watch’s data science readiness assessment.


How a Dedicated Data Warehouse Yields Better Insight than Your CRM or ERP

What percent of your enterprise data goes completely untapped? It’s far more than most organizations realize. Research suggests that as much as 68% of global enterprise data goes unused. The reasons are varied (we can get to the root cause with a current state assessment), but one growing problem stems from misconceptions about CRMs, ERPs, EHRs, and similar operational software systems.

The right operational software systems are valuable tools with their own effective reporting functions. The foundation of any successful reporting or analytics initiative depends on two factors: on a centralized source of truth that exists in a unified source format. All operational software systems struggle to satisfy either aspect of that criteria.

Believe it or not, one of the most strategic systems for data-driven decision-making is still a dedicated data warehouse. Here is the value a data warehouse brings to your organization and the necessary steps to implement that enhance your analytics’ accuracy and insight.

Download Now: Modern Data Warehouse Comparison Guide [Snowflake, Redshift, Azure Synapse, and Google BigQuery]

CRMs and ERPs Are Data Silos with Disparate Formats

Operational software systems are often advertised as offering a unified view, but that’s only true for their designed purpose. CRMs offer a comprehensive view of customers, ERPs of operations, and EHRs of patient or member medical history. Outside of their defined parameters, these systems are data silos.

In an HBR blog post, Edd Wilder-James captures the conundrum perfectly: “You can’t cleanly separate the data from its intended use. Depending on your desired application, you need to format, filter, and manipulate the data accordingly.”

Some platforms are enabled to integrate outside data sources, but even that provides you with a filtered view of your data, not the raw and centralized view necessary to generate granular and impactful reports. It’s the difference between abridged and unabridged books – you might glean chunks of the big picture but miss entire sections or chapters that are crucial to the overall story.

Building a dedicated data warehouse removes the question of whether your data sets are complete. You can extract, transfer, and load data from source systems into star schemas with a unified format optimized for business users to leverage. The data is formatted around the business process rather than the limitations of the tool. That way, you can run multifaceted reports or conduct advanced analytics when you need it – without anchoring yourself to any specific technology.

Tracking Down Your Data Sources

In all honesty, organizations not familiar with the process often overlook vital information sources. There might be a platform used to track shipping that only one member of your team uses. Maybe there’s a customer service representative who logs feedback in an ad hoc document. Or it’s possible there’s HIPAA-compliant software in use that isn’t automatically loading into your EHR. Regardless of your industry, there are likely gaps in your knowledge well outside of the CRMs, ERPs, EHRs, and other ostensibly complete data sources.

How do you build a single source of truth? It’s not as simple as shifting around a few sources. Implementing a dedicated data warehouse requires extensive planning and preparation. The journey starts with finding the invisible web of sources outside of your primary operational software systems. Those organizations that choose to forgo a full-fledged current state assessment to identify those hidden sources only achieve fragmentary analytics at best.

Data warehouse implementations need guidance and buy-in at the corporate level. That starts with a well-defined enterprise data strategy. Before you can create your strategy, you need to ask yourself questions such as these:

  • What are your primary business objectives?
  • What are your key performance indicators?
  • Which source systems contribute to those goals?
  • Which source systems are we currently using across the enterprise?

By obtaining the answers to these and other questions from decision-makers and end users, you can clarify the totality of your current state. Otherwise, hunting down those sources is an uphill battle.

Creating Data Warehouse Value that Lasts

Consolidating your dispersed data sources is just a starting point. Next, you need to extract the data from each source system and populate them within the data warehouse framework itself. A key component of this step is to test data within your warehouse to verify quality and completeness.

If data loss occurs during the ETL process, the impact of your work and veracity of your insights will be at risk. Running a variety of different tests (e.g., data accuracy, data completeness, data transformation, etc.) will reduce the possibility of any unanticipated biases in your single source of truth.

What about maintaining a healthy and dynamic data warehouse? How often should you load new data? The answer depends on the frequency of your reporting needs. As a rule of thumb, think in terms of freshness. If your data has gone stale by the time you’re loading it into your data warehouse, increase the frequency of your data refresh. Opt for real-time analytics if it will provide you with a strategic advantage, not because you want to keep current with the latest buzzword.

Improving Your Results with an Outsourced Partner

Each step in the process comes with its own complications. It’s easy to fall into common data warehousing pitfalls unless you have internal resources with experience pinpointing hidden data sources, selecting the right data model, and maintaining your data warehouse post-implementation.

One of our clients in the healthcare software space was struggling to transition to a dynamic data warehousing model that could enhance their sales. Previously, they had a reporting application that they were using on a semi-annual basis. Though they wanted to increase the frequency of their reporting and enable multiple users to run reports simultaneously, they didn’t have the internal expertise to confidently navigate these challenges.

Working with 2nd Watch made a clear difference. Our client was able to leverage a data warehouse architecture that provided daily data availability (in addition to the six-month snapshot) and self-service dashboards that didn’t require changes or updates on their part. We also set them on the right path to leverage a single source of the truth through future developments.

Our strategies in that project prioritized our client’s people instead of a specific technology. We considered the reporting and analytics needs of their business users rather than pigeonholing their business into a specific tool. Through our tech-agnostic approach, we guided them toward a future state that provided strategic advantage and a clear ROI that might have otherwise gone unachieved.

Want your data warehouse to provide you with a single source of the truth? Schedule a whiteboard session to review your options and consolidate your data into actionable insight.


What Real-Time Analytics Looks Like for Real-World Businesses

Real-time analytics. Streaming analytics. Predictive analytics. These buzzwords are thrown around in the business world without a clear-cut explanation of their full significance. Each approach to analytics presents its own distinct value (and challenges), but it’s tough for stakeholders to make the right call when the buzz borders on white noise.

Which data analytics solution fits your current needs? In this post, we aim to help businesses cut through the static and clarify modern analytics solutions by defining real-time analytics, sharing use cases, and providing an overview of the players in the space.

TL;DR

  • Real-time or streaming analytics allows businesses to analyze complex data as it’s ingested and gain insights while it’s still fresh and relevant.
  • Real-time analytics has a wide variety of uses, from preventative maintenance and real-time insurance underwriting to improving preventive medicine and detecting sepsis faster.
  • To get the full benefits of real-time analytics, you need the right tools and a solid data strategy foundation.

What is Real-Time Analytics?

In a nutshell, real-time or streaming analysis allows businesses to access data within seconds or minutes of ingestion to encourage faster and better decision-making. Unlike batch analysis, data points are fresh and findings remain topical. Your users can respond to the latest insight without delay.

Yet speed isn’t the sole advantage of real-time analytics. The right solution is equipped to handle high volumes of complex data and still yield insight at blistering speeds. In short, you can conduct big data analysis at faster rates, mobilizing terabytes of information to allow you to strike while the iron is hot and extract the best insight from your reports. Best of all, you can combine real-time needs with scheduled batch loads to deliver a top-tier hybrid solution.

Stream Analytics Overview Courtesy of Microsoft

Streaming Analytics in Action

How does the hype translate into real-world results? Depending on your industry, there is a wide variety of examples you can pursue. Here are just a few that we’ve seen in action:

Next-Level Preventative Maintenance

Factories hinge on a complex web of equipment and machinery working for hours on end to meet the demand for their products. Through defects or standard wear and tear, a breakdown can occur and bring production to a screeching halt. Connected devices and IoT sensors now provide technicians and plant managers with warnings – but only if they have the real-time analytics tools to sound the alarm.

Azure Stream Analytics is one such example. You can use Microsoft’s analytics engine to monitor multiple IoT devices and gather near-real-time analytical intelligence. When a part needs a replacement or it’s time for routine preventative maintenance, your organization can schedule upkeep with minimal disruption. Historical results can be saved and integrated with other line-of-business data to cast a wider net on the value of this telemetry data.

Real-Time Insurance Underwriting

Insurance underwriting is undergoing major changes thanks to the gig economy. Rideshare drivers need flexibility from their auto insurance provider in the form of modified commercial coverage for short-term driving periods. Insurance agencies prepared to offer flexible micro policies that reflect real-time customer usage have the opportunity to increase revenue and customer satisfaction.

In fact, one of our clients saw the value of harnessing real-time big data analysis but lacked the ability to consolidate and evaluate their high-volume data. By partnering with our team, they were able to create real-time reports that pulled from a variety of sources ranging from driving conditions to driver ride-sharing scores. With that knowledge, they’ve been able to tailor their micro policies and enhance their predictive analytics.

Healthcare Analytics

How about this? Real-time analytics saves lives. Death by sepsis, an excessive immune response to infection that threatens the lives of 1.7 million Americans each year, is preventable when diagnosed in time. The majority of sepsis cases are not detected until manual chart reviews conducted during shift changes – at which point, the infection has often already compromised the bloodstream and/or vital tissues. However, if healthcare providers identified warning signs and alerted clinicians in real time, they could save multitudes of people before infections spread beyond treatment.

HCA Healthcare, a Nashville-based healthcare provider, undertook a real-time healthcare analytics project with that exact goal in mind. They created a platform that collects and analyzes clinical data from a unified data infrastructure to enable up-to-the-minute sepsis diagnoses. Gathering and analyzing petabytes of unstructured data in a flash, they are now able to get a 20-hour early warning sign that a patient is at risk of sepsis. Faster diagnoses results in faster and more effective treatment.

That’s only the tip of the iceberg. For organizations in the healthcare payer space, real-time analytics has the potential to improve member preventive healthcare. Once again, real-time data from smart wearables, combined with patient medical history, can provide healthcare payers with information about their members’ health metrics. Some industry leaders even propose that payers incentivize members to make measurable healthy lifestyle choices, lowering costs for both parties at the same time.

Getting Started with Real-Time Analysis

There’s clear value produced by real-time analytics but only with the proper tools and strategy in place. Otherwise, powerful insight is left to rot on the vine and your overall performance is hampered in the process. If you’re interested in exploring real-time analytics for your organization, contact us for an analytics strategy session. In this session lasting 2-4 hours, we’ll review your current state and goals before outlining the tools and strategy needed to help you achieve those goals.


A CTO’s Guide to a Modern Data Platform: Data Strategy and Governance

In our previous blog post on how to build a data warehouse in 6-8 weeks, we showed you how to get lightning-fast results and effectively create a working data warehouse with Snowflake. Future state integrations and governance needs are coming, though. This is why 2nd Watch highly recommends executing a data strategy and governance project in parallel with your Snowflake proof-of-concept. Knowing how to leverage Snowflake’s strengths to avoid common pitfalls will save you time, money, and re-work.

Consider one company that spent a year using the data discovery layer-only approach. With data sources all centralized in the data warehouse and all transformations occurring at run-time in the BI tool, the data team was able to deliver a full analytical platform to its users in less time than ever before. Users were happy, at first, until the logic became more mature and more complex and ultimately required more compute power (translating to higher cost) to keep the same performance expectations. For some, however, this might not be a problem but an expected outcome.

For this company, enabling analytics and reporting was the only need for the first year, but integration of data across applications was coming full steam ahead. The primary line of business applications needed to get near-real-time updates from the others. For example, marketing automation didn’t rely 100% on humans; it needed data to execute its rules, from creating ad campaigns to sending email blasts based on events occurring in other systems.

This one use case poked a big hole in the architecture – you can’t just have a data warehouse in your enterprise data platform. There’s more to it. Even if it’s years away, you need to effectively plan for it or you’ll end up in a similar, costly scenario. That starts with data strategy and governance.

ETL vs. ELT in Snowflake

Identify where your transformations occur and how they impact your downstream systems.

The new paradigm is that you no longer need ETL (Extract, Transform, Load) – you need ELT (Extract, Load, Transform). This is true, but sometimes misleading. Some will interpret ELT as no longer needing to build and manage the expensive pipelines and business logic that delay speed-to-insight, are costly to maintain, and require constant upkeep for changing business rules. In effect, it’s interpreted as removing the “T” and letting Snowflake solve for this. Unfortunately, someone has to write the code and business logic, and it’s best to not have your business users trying to do this when they’re better served working on your organization’s core goals.

In reality, you are not removing the “T” – you are moving it to a highly scalable and performant database after the data has been loaded. This is still going to require someone to understand how your customer data in Salesforce ties to a customer in Google Analytics that corresponds to a sale in your ERP. You still need someone who knows both the data structures and the business rules. Unfortunately, the “T” will always need a place to go – you just need to find the right place.

Ensure your business logic is defined only once in the entire flow. If you’ve written complex transformation code to define what “customer” means, when that business logic inevitably changes, you’ll be guaranteed that this definition of “customer” will flow the same way to your BI users as it does to your ERP and CRM. When data science and machine learning enter the mix, you’ll also avoid time spent in data prep and instead focus on delivering predictive insights.

You might be thinking that this all sounds even more similar to the data warehouse you’ve already built and are trying to replace. There’s some good news: Snowflake does make this easier, and ELT is still exactly the right approach.

Defining and Adjusting the Business Logic and Views

Snowflake enables an iterative process of data discovery, proof-of-concept, business value, and long-term implementation.

Perhaps you’ve defined a sales hierarchy and a salesperson compensation metric. The developer can take that logic, put it into SQL against the raw data, and refresh the dashboard, all while the business user is sitting next to them. Is the metric not quite what the user expected, or is the hierarchy missing something they hadn’t thought of in advance? Tweak the SQL in Snowflake and refresh. Iterate like this until the user is happy and signs off, excited to start using the new dashboard in their daily routine.

By confirming the business logic in the salesperson compensation example above, you’ve removed a major part of what made ETL so painful in the past: developing, waiting for a load to finish, and showing business users. That gap between load finishing and the next development cycle is a considerable amount of lost time and money. With this approach, however, you’ve confirmed the business logic is correct and you have the SQL already written in Snowflake’s data discovery views.

Developing your initial logic in views in Snowflake’s data discovery layer allows you to validate and “certify” it for implementation into the physical model. When you’ve completed the physical path, you can change the BI tool for each completed subject area to point to the physical layer instead of the data discovery layer.

If you have any questions about data strategy and governance, or if you want to learn more about how Snowflake can fit into your organization, contact us today.

This blog originally appeared as a section of our eBook, “Snowflake Deployment Best Practices: A CTO’s Guide to a Modern Data Platform.” Click here to download the full eBook.

Related Content:

What is Snowflake, How is it Different, and Where Does it Fit in Your Ecosystem?

How to Build a Data Warehouse in 6-8 Weeks

Methods to Implement a Snowflake Project


3 Options for Getting Started with a Modern Data Warehouse

In previous blog posts, we laid out the benefits of a modern data warehouse, explored the different types of modern data warehouses available, and discussed where a modern data warehouse fits in your overall data architecture.

Download Now: Modern Data Warehouse Comparison Guide [Snowflake, Amazon Redshift, Azure Synapse, and Google BigQuery]

There is no such thing as a one-size-fits-all data warehouse. To that end, there is no singular approach to getting started. Getting started depends on your goals, needs, and where you are today. In this blog post, we’ll outline a few options 2nd Watch offers for getting started with a modern data warehouse and the details for each.

  • Option 1: Data Architecture Whiteboard Session
  • Option 2: Modern Data Warehouse Strategy Session
  • Option 3: Modern Data Warehouse Quickstart

Option 1: 60-Minute Data Architecture Assessment

A 60-minute data architecture assessment is a great option to see how a modern data warehouse would fit in your current environment and what would be involved to get from where you are now to where you want to be.

During this session, we will outline a plan to achieve your goals and help you understand the tools, technologies, timeline, and cost to get there.

Who is this for? Organizations in the very early planning stages

Duration: 60 minutes

More Information

Option 2: Modern Data Warehouse Strategy

In order to see ROI and business value from your modern data warehouse, you must have a clear plan on how you are going to use it. During a modern data warehouse strategy project, our team will work with your stakeholders to understand your business goals and design a tech strategy to ensure business value and ROI from your data environment.

Who is this for? Organizations in the early planning stages looking to establish the business use case, cost benefits, and ROI of a modern data warehouse before getting started

Duration: 2-, 4-, 6-, or 8-week strategies are available

More Information

Option 3: Modern Data Platform Quickstart

You have your strategy laid out and are ready to get started ASAP. The modern data platform quickstart is a great option to get your modern data warehouse up and running in as few as six weeks.

During this quickstart, we’ll create a scalable data warehouse; clean, normalize, and ingest data; and even provide reports for predefined use cases.

Who is this for? Organizations that have outlined their strategy and are ready to start seeing the benefits of a modern data warehouse

Duration: 6 weeks

More Information

Not sure where to begin? We recommend beginning with a 60-minute data architecture assessment. This session allows us to walk through your current architecture, understand your organization’s pain points and goals for analytics, brainstorm on a future state architecture based on your goals, and then come up with next steps. Furthermore, the assessment allows us to determine if your organization needs to make a change, what those changes are, and how you might go about implementing them. Simply put, we want to understand the current state, learn about the future state of what you want to build toward, and help you create a plan so you can successfully execute on a modern data warehouse project.

A Word of Warning

Modern data warehouses are a big step forward from traditional on-premise architectures. They allow organizations to innovate quicker and provide value to the business much faster. An organization has many options in the cloud and many vendors offer a cloud data warehouse, but be careful: building a modern data warehouse architecture is highly involved and may require multiple technologies to get you to the finish line.

The most important thing to do when embarking on a modern data warehouse initiative is to have an experienced partner to guide you through the process the right way from establishing why a cloud data warehouse is important to your organization to outlining what the future state vision should be to develop a plan to get you there.

Data warehouse architecture is changing, don’t fall behind your competition! With multiple options for getting started, there is no reason to wait.

We hope you found this information valuable. If you have any questions or would like to learn more, please contact us and we’ll schedule a time to connect.


Roadmap to an Insurance Analytics Solution: 4 Steps to Solve Business Problems with P&C Analytics

P&C insurance is an incredibly data-driven industry. Your company’s core assets are data, your business revolves around collecting data, and your staff is focused on using data in their day-to-day workstreams. Although data is collected and used in normal operations, oftentimes the downstream analytics process is painful (think of those month-end reports). This is for any number of reasons:

  • Large, slow data flows
  • Unmodeled data that takes manual intervention to integrate
  • Legacy software that has a confusing backend and user interface
  • And more

Creating an analytics ecosystem that is fast and accessible is not a simple task, but today we’ll take you through the four key steps 2nd Watch follows to solve business problems with an insurance analytics solution. We’ll also provide recommendations for how best to implement each step to make these steps as actionable as possible.

Step 1: Determine your scope.

What are your company’s priorities?

  • Trying to improve profit margin on your products?
  • Improving your loss ratio?
  • Planning for next year?
  • Increasing customer satisfaction?

To realize your strategic goals, you need to determine where you want to focus your resources. Work with your team to find out which initiative has the best ROI and the best chance of success.

First, identify your business problems.

There are so many ways to improve your KPIs that trying to identify the best approach can very quickly become overwhelming. To give yourself the best chance, be deliberate about how you go about solving this challenge.

What isn’t going right? Answer this question by talking to people, looking at existing operational and financial reporting, performing critical thinking exercises, and using other qualitative or quantitative data (or both).

Then, prioritize a problem to address.

Once you identify the problems that are impacting metrics, choose one to address, taking these questions into account:

  • What is the potential reward (opportunity)?
  • What are the risks associated with trying to address this problem?
  • How hard is it to get all the inputs you need?

RECOMMENDATION
Taking on a scope that is too large, too complex, or unclear will make it very difficult to achieve success. Clearly set boundaries and decide what is relevant to determine which pain point you’re trying to solve. A defined critical path makes it harder to go off course and helps you keep your goal achievable.

Step 2: Identify and prioritize your KPIs.

Next, it’s time to get more technical. You’ve determined your pain points, but now you must identify the numeric KPIs that can act as the proxies for these business problems.

Maybe your business goal is to improve policyholder satisfaction. That’s great! But what does that mean in terms of metrics? What inputs do you actually need to calculate the KPI? Do you have the data to perform the calculations?

Back to the example, here are your top three options:

Based on this information, even though the TTC metric may be your third-favorite KPI for measuring customer satisfaction, the required inputs are identified and the data is available. This makes it the best option for the data engineering effort at this point in time. It also helps you identify a roadmap for the future if you want to start collecting richer information.

RECOMMENDATION
As you identify the processes you’re trying to optimize, create a data dictionary of all the measures you want to use in your reporting. Appreciate that a single KPI might:

  • Have more and higher quality data
  • Be easier to calculate
  • Be used to solve multiple problems
  • Be a higher priority to the executive team

Use this list to prioritize your data engineering effort and create the most high-value reports first. Don’t engineer in a vacuum (i.e., generate KPIs because they “seem right”). Always have an end business question in mind.

Step 3: Design your solution.

Now that you have your list of prioritized KPIs, it’s time to build the data warehouse. This will allow your business analysts to slice your metrics by any number of dimensions (e.g., TTC by product, TTC by policy, TTC by region, etc.).

2nd Watch’s approach usually involves a star schema reporting layer and a customer-facing presentation layer for analysis. A star schema has two main components: facts and dimensions. Fact tables contain the measurable metrics that can be summarized. In the TTC example, the fact-claim tables might contain a numeric value containing the number of days to close a claim. A dimension table would then provide context for how you pivot the measure. For example, you might have a dimension-policyholder table that contains attributes to “slice” the KPI value (e.g., policyholder age, gender, tenure, etc.).

Once you design the structure of your database design, you can build it. This involves transforming the data from your source system to the target database. You’ll want to consider the ETL (extract-transform-load) tool that will automate this transformation, and you’ll also need to consider the type of database that will be used to store your data. 2nd Watch can help with all these technology decisions.

You may also want to take a particular set of data standards into account, such as the ACORD Standards, to ensure more efficient and effective flow of data across lines of business, for example. 2nd Watch can take these standards into account when implementing an insurance analytics solution, giving you confidence that your organization can use enterprise-wide data for a competitive advantage.

Finally, when your data warehouse is up and running, you want to make sure your investment pays off by managing the data quality of your data sources. This can all be part of a data governance plan, which includes data consistency, data security, and data accountability.

RECOMMENDATION
Don’t feel like you need to implement the entire data warehouse at once. Be sure to prioritize your data sources and realize you can gain many benefits by just implementing some of your data sources.

Step 4: Put your insurance analytics solution into practice.

After spending the time to integrate your disparate data sources and model an efficient data warehouse, what do you actually get out of it? As an end business user, this effort can bubble up as flat file exports, dashboards, reports, or even data science models.

I’ve outlined three levels of data maturity below:

Level 1
The most basic product would be a flat file. Often, mid-to-large-sized organizations working with multiple source systems work in analytical silos. They connect directly to the back end of a source system to build analytics. As a result, intersystem analysis becomes complex with non-standard data definitions, metrics, and KPIs.

With all of that source data integrated in the data warehouse, the simplest way to begin to analyze the data is off of a single flat extract. The tabular nature of a flat file will also help business users answer basic questions about their data at an organizational level.

Level 2
Organizations farther along the data maturity curve will begin to build dashboards and reports off of the data warehouse. Regardless of your analytical capabilities, dashboards allow your users to glean information at a glance. More advanced users can apply slicers and filters to better understand what drives their KPIs.

By compiling and aggregating your data into a visual format, you make the breadth of information at your organization much more accessible to your business users and decision-makers.

Level 3
The most mature product of data integration would be data science models. Machine learning algorithms can detect trends and patterns in your data that traditional analytics would take a long time to uncover, if ever. Such models can help insurers more efficiently screen cases and predict costs with greater precision. When writing policies, a model can identify and manage risk based on demographic or historic factors to determine ROI.

RECOMMENDATION
Start simple. As flashy and appealing as data science can be to stakeholders and executives, the bulk of the value of a data integration platform lies in making the data accessible to your entire organization. Synthesize your data across your source systems to produce file extracts and KPI scorecards for your business users to analyze. As users begin to adopt and understand the data, think about slowly scaling up the complexity of analysis.

Conclusion

This was a lot of information to absorb, so let’s summarize the roadmap to solving your business problems with insurance analytics:

  • Step 1: Determine your scope.
  • Step 2: Identify and prioritize your KPIs.
  • Step 3: Design your solution.
  • Step 4: Put your insurance analytics solution into practice.

2nd Watch’s data and analytics consultants have extensive experience with roadmaps like this one, from outlining data strategy to implementing advanced analytics. If you think your organization could benefit from an insurance analytics solution, feel free to get in touch to discuss how we can help.


How Business and IT Should Partner to Build a Data Strategy

Any functioning company has a vision, goals, and direction. A successful company follows a strategy – set forth by its executives – to realize its vision, achieve its goals, and determine its direction. Executing executive strategy requires cross-functional collaboration amongst teams across the organization.

Having a data strategy is crucial to achieving your greater business goals. To set attainable and tangible business objectives, you need to utilize data. Your IT department will implement and oversee the technologies powering your data strategy; therefore, business processes and goals must factor in IT resources and capabilities. Your organization’s business unit should work closely with your IT team to build an effective data strategy.  

Utilizing Data Analysis to Pursue Strategies

When it comes to business goals, each goal should have a quantitative KPI or metric that is extracted from data and analytics. Of course, this is easier said than done! Even the simplest of business questions can spawn dozens of data threads needed for analysis. For example, if you are looking to track business profitability, you’ll have to look at key components such as revenue, COGS, and overhead. Each of these data sources must be broken down even further to fully understand the several metrics that make up the greater question of profitability. The ultimate goal of having a data strategy is to define all the inputs required to calculate key metrics and incorporate them into your production processes.

Your Data Strategy Checklist

Before engaging in any data management project, it’s important to outline your to-do list. Below is a checklist with the foundational steps to building out a data strategy:

  • Scope the project (what business problem are you trying to solve?)
  • Generate a work breakdown structure for strategy.
  • Create a timeline with milestones.
  • Define teams and individual role definitions.
  • Determine how you collaborate with others.
  • Schedule meetings (kick-off, status, sprint planning, and steering meetings). 
  • Begin research of your technology stack.
  • Budget for your data strategy project.
  • Kick off with the team!

Let’s Get Started!

As you work toward building a data strategy, there are four deliverables you should focus on: a current state understanding, a future state architecture, a gap analysis, and a road map.

Current State Understanding

Your data strategy should reinforce and advance your overall business strategy, which refers to the processes you use to operate and improve your business. To grow, you must establish your baseline to measure your growth. To gain an understanding of your current state, review your current data architecture, data flows, and source systems by:

  • Collecting existing diagrams and analyzing existing discovery
  • Verifying source systems (you have more than you think!)
  • Diagramming and visualizing
  • Having a logical model (what are the existing business domains?)

Future State Architecture

Once you’ve established a baseline, imagine where you’d like the future state architecture. Your future state architecture should support the data platform at your company, including a logical model of the data warehouse and technology decisions. To determine your future state, define your data threads, which requires you to identify your source systems and domains. 

Gap Analysis

What changes need to happen to move your company from the current state to the desired future state? Establish your technical framework and identify the gaps. This framework should be built for moving, curating, and delivering data to your business use cases. 

Road Map

Roading mapping is the most time-consuming and the most important deliverable. Your data strategy roadmap will outline how and when to implement your data vision and mission. This will help you manage change and ensure that IT goals are aligned with business goals. A well-thought-out roadmap reveals true dependencies so you can adjust as priorities shift. There are several important steps to building out a road map, which we will cover below. 

Step 1: Build and estimate a backlog.

Your backlog should be based on your intended future state and what business deliverables you want to generate.

Questions to answer:

  • What data threads need to be engineered?
  • How are you going to deliver the inputs to the artifact?
  • How are you going to productionalize the result to create the action?

Step 2: Define your vision and interim goals.

Before you can embark on any project, you’ll need to establish your end goals and the goals along the way.

Questions to answer:

  • What does success look like?
  • What priorities should you target?

Step 3: Identify your KPIs.

Determine the data and metrics that will be the most useful for operational and strategic decisions.

Questions to answer:

  • Who needs what information?
  • How valuable is that information to guide your decisions?

Step 4: Understand everything about your data.

This means understanding your sources of data, data relationships, and data quality issues.

Questions to answer:

  • What data are you sourcing from?
  • What data issues can be resolved by code versus process changes?
  • How can you map this data to a new, clean, holistic data model for reporting and analytics?

Step 5: Recommend a selection of technologies and tools.

Technologies and tools will be the means to your end, so be sure to conduct a thorough evaluation process. 

Questions to answer:

  • What architecture considerations are required to best use your technology stack and ecosystem?
  • What orchestration tools will you need?
  • What additional tools will you need to consider in the future as the platform matures?

Step 6: Design a high-level solution architecture.

Use this architecture on your data marts, leading to quicker access to data and faster insights. 

Questions to answer:

  • How should raw data, data collection, data storage, and access be structured for ease of use?
  • How frequently – and with what method – should this data be refreshed?
  • Who will own and support this in the future?

Step 7: Generate a specific backlog of tasks for implementing o9 development.

Questions to answer:

  • How will you build out the end-to-end foundation?
  • How will you coordinate with groups to gain the most immediate benefit?

Step 8: Develop a go-forward plan with a budget, milestones, and a timeline.

These components of your roadmap will make your future state a reality.

Questions to answer:

  • When can you complete this project?
  • How will you allocate the work?
  • How much will it all cost?
  • What internal and external resources are needed to put everything into action?

A Timeline of Your Data Strategy Road Map

At 2nd Watch, our approach is based on a 5- to 7-month data strategy timeline. Our data experts and consultants think about implementing a data strategy in phases. 

Phase 1: Foundation and process – 3 to 4 months

  • Set up your ETL and audit framework.
  • Ingest your primary data set into a persistent storage layer.
  • Engineer a business model for Phase 1 scope.
  • Create a data dictionary, source to target, and support documentation for scaling the platform.
  • Develop business use cases based on Phase 1 scope.
  • Establish a strategic foundation for ongoing maturity and add future data sources.
  • Assign a data engineer and architect to support model development.

Phase 2: Development analytics – 2 to 3 months

  • Ingest your primary data sources for development.
  • Add integration dataflows to increase and expand the business layer for Phase 2 scope.
  • Identify SMEs and data champions to drive QA and identify new report requirements.

Phases 3 and 4: Secondary sources

  • Ingest all other data sources to supplement data warehouse development.
  • Add integration dataflows to increase and expand the business layer for Phase 3 and 4 scope.
  • Integrate new dataflows into existing reporting and create opportunities for new audiences and enterprise level roll-ups.

Phase 5 and beyond: Support

  • Continue to model data for Phases 5 and beyond.
  • Expand advanced analytics capabilities for predictive and clustering models.
  • Eliminate downstream dependencies on legacy systems to allow sunsetting applications.
  • Integrate additional data consumers and data applications for expanded functionality.

2nd Watch’s consultants work with your team to understand your business goals and design a tech strategy to ensure success throughout the organization. From planning to technology selection to user adoption, our team’s data advisory services enhance business optimization via data-driven insights for the long haul. Contact us today to learn how 2nd Watch can build a modern data foundation for your business intelligence and predictive analytics to ensure your organization can make better decisions now and in the future.