Holistic Healthcare Clinic
A Modern Data Warehouse for Holistic Healthcare Analytics.
A healthcare client wanted to provide a holistic view for healthcare customers and their internal team that included insightful reports and dashboards.
Their current set-up meant data was difficult to analyze, unreliable, and slow to access.
2nd Watch integrated, standardized, and centralized the client’s data sources to a scalable modern data warehouse, enabling high-performance reports and insightful dashboards.
Our healthcare client was looking for a way to improve executive customer reporting to find meaningful information and insights in their data. The solution needed to address data integration challenges, alleviate manual process, and provide the ability to scale as the business grows.
With a strong foundation and experience in healthcare data management, analytics, and data science, this client chose 2nd Watch to create a modern data warehouse for healthcare.
Multiple Disparate MySQL Databases with Non-Standardized Data
Prior to this project, our client had a separate MySQL database for each of their customers, living across different MySQL hosts. Each database was easily customized by their customers, leading to KPIs and naming conventions that were not standardized. Our client wanted to be able to analyze data across multiple different customer segments, but this was unobtainable with their current set-up.
Additionally, these MySQL databases were constantly receiving data from our client’s application, causing fear of system shutdowns and slow speeds when trying to access the data.
In order to pave the way for the executive customer reports they needed, their data needed to be integrated, standardized, and centralized to a modern data warehouse.
Centralizing and Standardizing Disparate Customer Data Sources for a Holistic View
The first part of this project was to create a centralized modern data warehouse in Snowflake to allow for analysis across customers, with the added benefits of standardizing KPIs and increasing the speed at which reports can be generated. Airflow was used to orchestrate a data migration pipeline to ingest data from MySQL and transfer to Snowflake. By using Airflow, our ingestion process was able to transfer data from hundreds of tables for hundreds of databases all in parallel, allowing for a fast and efficient transfer of data. Airflow was used to orchestrate our data warehouse by inserting any new or updated data into our data warehouse tables.
Once the data warehouse was created, we also implemented reports with row-level security for customers to have access to their data and their data alone. This allowed our client to share the data with their customers adding value to both parties.
2nd Watch was able to develop a comprehensive solution that created a centralized modern data platform for our client. This allowed our client to access their data without worrying about system shutdowns and access it at much greater speeds. Additionally, our client was able to monetize their new modern data warehouse for healthcare by creating an analytics platform that their customers could access from this data. Ultimately, the increase in data access and availability has allowed our client and their customers to better analyze and understand their data with the goal of providing a better quality of care to patients.