A CTO’s Guide to a Modern Data Platform: What is Snowflake, How is it Different, and Where Does it Fit in Your Ecosystem?

Chances are, you’ve been here before – a groundbreaking new data and analytics technology has started making waves in the market, and you’re trying to gauge the right balance between marketing hype and reality. Snowflake promises to be a self-managing data warehouse that can get you speed-to-insight in weeks, as opposed to years. Does Snowflake live up to the hype? Do you still need to approach implementation with a well-defined strategy? The answer to both of these questions is “yes.”

What Is Snowflake and How Is It Different?

Massive scale….Low overhead

Snowflake is one of the few enterprise-ready cloud data warehouses that brings simplicity without sacrificing features. It automatically scales, both up and down, to get the right balance of performance vs. cost. Snowflake’s claim to fame is that it separates compute from storage. This is significant because almost every other database, Redshift included, combines the two together, meaning you must size for your largest workload and incur the cost that comes with it.

With Snowflake, you can store all your data in a single place and size your compute independently. For example, if you need near-real-time data loads for complex transformations, but have relatively few complex queries in your reporting, you can script a massive Snowflake warehouse for the data load, and scale it back down after it’s completed – all in real time. This saves on cost without sacrificing your solution goals.

Elastic Development and Testing Environments

Development and testing environments no longer require duplicate database environments. Rather than creating multiple clusters for each environment, you can spin up a test environment as you need it, point it at the Snowflake storage, and run your tests before moving the code to production. With Redshift, you’re feeling the maintenance and cost impact of three clusters all running together. With Snowflake, you stop paying as soon as your workload finishes because Snowflake charges by the second.

With the right DevOps processes in place for CI/CD (Continuous Integration/Continuous Delivery), testing each release becomes closer to a modern application development approach than it does a traditional data warehouse. Imagine trying to do this in Redshift.

Avoiding FTP with External Data Sharing

The separated storage and compute also enables some other differentiating features, such as data sharing. If you’re working with external vendors, partners, or customers, you can share your data, even if the recipient is not a Snowflake customer. Behind the scenes, Snowflake is creating a pointer to your data (with your security requirements defined). If you commonly write scripts to share your data via FTP, you now have a more streamlined, secure, and auditable path for accessing your data outside the organization. Healthcare organizations, for example, can create a data share for their providers to access, rather than cumbersome manual processes that can lead to data security nightmares.

Where Snowflake Fits Into Your Ecosystem

Snowflake is a part of Your Data Ecosystem, but It’s not in a Silo.

Always keep this at the top of your mind. A modern data platform involves not only analytics, but application integration, data science, machine learning, and many other components that will evolve with your organization. Snowflake solves the analytics side of the house, but it’s not built for the rest.

When you’re considering your Snowflake deployment, be sure to draw out the other possible components, even if future tools are not yet known. Knowing which Snowflake public cloud flavor to choose (Azure or AWS) will be the biggest decision you will make. Do you see SQL Server, Azure ML, or other Azure PaaS services in the mix; or is the AWS ecosystem more likely to fit better in the organization?

As a company, Snowflake has clearly recognized that they aren’t built for every type of workload. Snowflake partnered with Databricks to allow heavy data science and other complex workloads to run against your data. The recent partnership with Microsoft will ensure Azure services continue to expand their Snowflake native integrations – expect to see a barrage of new partnership announcements during the next 12 months.

If you have any questions or want to learn more about how Snowflake can fit into your organization, contact us today.

 


Ready to Migrate your Data to the Cloud? Answer these 4 Questions to find Out

Many companies are already storing their data in the cloud and even more are considering making the migration to the cloud. The cloud offers unique benefits for data access and consolidation, but some businesses choose to keep their data on-prem for various reasons. Data migration isn’t a one size fits all formula, so when developing your data strategy, think about your long-term needs and goals for optimal results.

We recommend evaluating these 4 questions before making the decision to migrate your data to the cloud:

1. Why do you Want to Migrate your Data to the Cloud?

Typically, there are two reasons businesses find themselves in a position of wanting to change their IT infrastructure. Either your legacy platform is reaching end of life (EOL) and you’re forced to make a change, or it’s time to modernize. If you’re faced with the latter – your business data expanded beyond the EOL platform – it’s a good indication migrating to the cloud is right for you. The benefits of cloud-based storage can drastically improve your business agility.

2. What is Important to You?

You need to know why you’re choosing the platform you are deploying and how it’s going to support your business goals better than other options. Three central arguments for cloud storage – that are industry and business agnostic – include:

  • Agility: If you need to move quickly (and what business doesn’t?), the cloud is for you. It’s easy to start, and you can spin up a cloud environment and have a solution deployed within minutes or hours. There’s no capital expense, no server deployment, and no need for an IT implementation team.
  • Pay as you go: If you like starting small, testing things before you go all in, and only paying for what you use, the cloud is for you. It’s a very attractive feature for businesses hesitant to move all their data at once. You get the freedom and flexibility to try it out, with minimal financial risk. If it’s not a good fit for your business, you’ve learned some things, and can use the experience going forward. But chances are, the benefits you’ll find once utilizing cloud features will more than prove their value.
  • Innovation: If you want to ride the technology wave, the cloud is for you. Companies release new software and features to improve the cloud every day, and there’s no long release cycles. Modernized technologies and applications are available as soon as they’re released to advance your business capabilities based on your data.

3. What is your Baseline?

The more you can plan for potential challenges in advance, the better. As you consider data migration to the cloud, think about what your data looks like today. If you have an on-prem solution, like a data warehouse, lift and shift is an attractive migration plan because it’s fairly easy.

Many businesses have a collection of application databases and haven’t yet consolidated their data. They need to pull the data out, stage it, and store it without interfering with the applications. The main cloud providers offer different, but similar options to get your data into a place where it can be used. AWS offers S3, Google Cloud has Cloud Storage, and Azure provides Blob storage. Later, you can pull the data into a data warehousing solution like AWS Redshift, Google BigQuery, Microsoft Synapse, or Snowflake.

4. How do you Plan to use your Data?

Always start with a business case and think strategically about how you’ll use your data. The technology should fit the business, not the other way around. Once you’ve determined that, garner the support and buy-in of sponsors and stakeholders to champion the proof of concept. Bring IT and business objectives together by defining the requirements and the success criteria. How do you know when the project is successful? How will the data prove its value in the cloud?

As you move forward with implementation, start small, establish a reasonable timeline, and take a conservative approach. Success is crucial for ongoing replication and investment. Once everyone agrees the project has met the success criteria, celebrate loudly! Demonstrate the new capabilities, and highlight overall business benefits and impact, to build and continue momentum.

Be Aware of your Limitations

When entering anything unknown, remember that you don’t know what you don’t know. You may have heard things about the cloud or on-prem environments anecdotally, but making the decision of when and how to migrate data is too important to do without a trusted partner. You risk missing out on big opportunities, or worse, wasting time, money, and resources without gaining any value.

2nd Watch is here to serve as your trusted cloud advisor, so when you’re ready to take the next step with your data, contact Us.

Learn more about 2nd Watch Data and Analytics services

-Sam Tawfik, Sr Product Marketing Manager, Data & Analytics


Migrating Data to Snowflake – An Overview

When considering migrating your data to the cloud, everyone’s familiar with the three major cloud providers – AWS, Google Cloud, and Microsoft Azure. But there are a few other players you should also take note of. Snowflake is a leading cloud data platform that offers exceptional design, scalability, simplicity, and return on investment (ROI).

What is Snowflake?

The Snowflake cloud data platform was born in the cloud for data warehousing. It’s built entirely to maximize cloud usage and designed for almost unlimited scalability. Users like the simplicity, and businesses gain significant ROI from the wide range of use cases Snowflake supports.

Out of the box, Snowflake is easy to interact with through its web interface. Without having to download any applications, users can connect with Snowflake and create additional user accounts for a fast and streamlined process. Additionally, Snowflake performs as a data platform, rather than just a data warehouse. Data ingestion is cloud native and existing tools enable effortless data migration.

Business Drivers

The decision to migrate data to a new cloud environment, or data warehousing solution, needs to be based on clearly defined value. Why are you making the transition? What’s your motivation? Maybe you need to scale up, or there’s some sort of division or business requirement for migration. Often times, companies have a particular implementation that needs to change, or they have specific needs that aren’t being met by their current data environment.

Take one of our clients, for instance. When the client’s company was acquired, they came to utilize a data warehouse shared by all the companies the acquiring company owned. When the client was eventually sold, they needed their own implementation and strategy for migrating data into the cloud. Together, we took the opportunity to evaluate some of the newer data platform tools, like Snowflake, for their specific business case and to migrate quickly to an independent data platform.

With Snowflake, set up was minimal and supported our client’s need for a large number of database users. Migrating from the shared data warehouse to Snowflake was relatively easy, and it gave all users access through a simple web interface. Snowflake also provided more support for unstructured data usage, which simplified querying things like JSON or nested data.

Implementation

Migrating data to Snowflake is generally a smooth transition because Snowflake accepts data from your existing platform. For instance, if data is stored in Amazon S3, Google Cloud, or Azure, you can create Snowflake environments in each then ingest the data using SQL commands and configuration. Not only can you run all the same queries with minor tweaks and get the same output, but Snowflake also fits additional needs and requirements. If you’ve worked in SQL in any manner – on an application database, or in data warehousing – training is minimal.

Another advantage with Snowflake is its ability to scale either horizontally or vertically to pull in any amount of data. And since it is cloud native, Snowflake has embraced the movement toward ‘pay as you go’ – in fact, that’s their entire structure. You only pay for the ingestion time and when the data warehouse is running. After that, it shuts off, and so does your payment. Cost-effective implementation lets you experiment, compare, test, and iterate on the best way to migrate each piece of your data lifecycle.

Long Term Results

Snowflake has yielded successful data migrations with users because of its ease of use and absence of complications. Users also see performance improvements because they’re able to get their data faster than ever and they can grow with Snowflake, bringing in new and additional data sources and tools, taking advantage of artificial intelligence and machine learning, increasing automation, and experimenting and iterating.

From a security and governance perspective, Snowflake is strong. Snowflake enforces a multi-layer security structure, including user management. You can grant access to certain groups, organize them accordingly, integrate with your active directory, and have it run with those permissions. You assign an administrator to regulate specific accessibility for tables in specified areas. Snowflake also lets you choose your desired security level during implementation. You have the option of enterprise level, HIPAA compliance, and a maximum security level with a higher rate per second.

Do you want to explore data migration opportunities? Make the most of your data by partnering with trusted experts. We’re here to help you migrate, store, and utilize data to grow your business and streamline operations. If you’re ready to the next step in your data journey, Contact Us.

Learn more about 2nd Watch Data and Analytics services

-Sam Tawfik, Sr Product Marketing Manager, Data & Analytics


Cloud Crunch Podcast: Data, AI & ML on Google Cloud

If you’re trying to run your business smarter, not harder, chances are you’re utilizing data to gain insights into the decision-making process and gain a competitive advantage. In the latest episode of our podcast, we talk with data and AI & ML expert, Rui Costa at Google Cloud, about why and when to use cloud data offerings and how to make the most of your data in the cloud. Listen now on Spotify, iTunes, iHeart Radio, Stitcher, or wherever you get your podcasts.

We’d love to hear from you! Email us at CloudCrunch@2ndwatch.com with comments, questions and ideas.


Cloud Crunch Podcast: Data Migration to Snowflake

There’s a new(ish) major player in cloud data management! 2nd Watch data and analytics experts, Sam Tawfik and Spencer Dorway, talk about practical data migration to Snowflake. Listen now on Spotify, iTunes, iHeart Radio, Stitcher, or wherever you get your podcasts.

We’d love to hear from you! Email us at CloudCrunch@2ndwatch.com with comments, questions and ideas.