Even though public cloud adoption has become mainstream amongenterprises, the heavily touted full cloud adoption has not become a reality for many companies, nor will it for quite some time. Instead we see greater adoption of hybrid cloud, a mixture of public and private clouds, as the predominant deployment of IT services. With private cloud deployments largely consisting of market share leader, VMware, it gives even more credence to a VMware Cloud on AWS solution.
Looking back 2 years to when VMware and AWS made the announcement that they had co-engineered a cloud solution, it makes a lot more sense, now. That wasn’t necessarily always the case. I’ll be among the first to admit that I failed to see how the two competitive solutions would coexist in a way that provided value to the customer. But then again, I was fully drinking the cloud punch that said refactoring applications and deploying in a “cattle vs pets” mentality was necessary to enable a full-on digital transformation to merely survive in the evolving aaS world.
What I was not considering was that more than 75% of private clouds were running on VMware. Or that companies had made a significant investment into not only the licensing and tooling, but also in their people, to run VMware. It would not have made sense to move everything to the cloud in many situations.
I viewed it solely as a “lift and shift” opportunity. It provided a means for companies to move their IT infrastructure out of the data center and “check the box” for fully migrating to the cloud while allowing for the gradual adoption of AWS cloud native solutions as they trained staff accordingly.
While it is true that performing a complete data center evacuation is a common request with various factors influencing the decision, delaying cloud native is less of a driver. Some companies are making the decision because they have been unsuccessful in renegotiating their contract with their colo-provider and find themselves in a tough situation resulting in the need to rapidly move or be locked-in for another lengthy contract. In other situations, the CIO has decided that their valuable human capital would be better spent delivering higher value to their company as opposed to running a data center and converting from a CAPEX to OPEX model for their IT infrastructure works better for their business.
However, there are two use cases that seem to be bigger drivers of VMware Cloud on AWS; the need for improved performance and disaster recovery.
Aside from on-demand access to infrastructure, another big advantage of AWS is the sheer number of solutions they have created that become available to use in a matter of minutes and can be easily connected to your applications residing on VMware VMs. With VMware Hybrid Cloud Extension (HCX), moving applications between on-Premises VMware deployments and VMware Cloud on AWS deployments is seamless. This allows your VMs be closer to the dependent AWS tooling to improve latency and may result in improved performance for your users. If you have a geographically disbursed user base, you can easily set up a VMware Cluster in a region much closer, further reducing latency.
I do want to caution, though, that prior to performing a migration of your applications to VMware Cloud on AWS, you should create a dependency map of all your VMs in your on-premises environment. It is necessary to have a thorough understanding of what other VMs your applications are communicating with. We have seen numerous cases where proper identification of dependencies has not occurred, resulting is dissatisfaction when the application is moved to VMware Cloud on AWS but the SAP database remains on-premises. So, while you may have brought the application closer to your users, performance could be impacted if the dependencies are not located nearby.
The other use case that has been gaining adoption is the ability to have a disaster recovery environment. With the severity of natural disasters occurring at what seems like an increased rate, there is a real threat that your business could be impacted with downtime. VMware Cloud on AWS coupled with VMware Site Recovery Manager provides you an opportunity to put in place a business continuity plan in geographically diverse regions to help ensure that your business keeps running.
The other exciting thing is that hybrid cloud no longer has to be located outside your data center. VMware Cloud on AWS has gained such wide spread acceptance that, at AWS re:Invent 2019, VMware announced the opening of a VMware Cloud on AWS Outposts Beta program, which brings the popular features of AWS Cloud right into your data center to work alongside VMware. This seems like it would be best for clients who need the benefits of VMware Cloud on AWS but have some data sovereignty issues or legacy applications that simply cannot migrate to off premise VMware Cloud.
As one of only a handful of North American VMware Partners to possess the VMware Master Services Competency in VMware Cloud on AWS, 2nd Watch has performed numerous successful VMware Cloud on AWS Implementations. We also support AWS Outposts, helping AWS customers overcome challenges that exist due to managing and supporting infrastructures both on-premises and in cloud environments, for a truly consistent hybrid experience.
If you want to understand how VMware Cloud on AWS can further enable your hybrid cloud adoption, schedule a VMware Cloud on AWS Workshop – a 4-hour, complimentary, on-site overview of VMware Cloud on AWS and appropriate use cases – to see if it is right for your business.
AWS re:Invent 2019 has come and gone, and now the collective audience has to sort through the massive list of AWS announcements released at the event. According to the AWS re:Invent 2019 Recap communication, AWS released 77 products, features and services in just 5 days! Many of the announcements were in the Machine Learning (ML) space (20 total), closely followed by announcements around Compute (16 total), Analytics (6 total), Networking and Content Delivery (5 total), and AWS Partner Network (5 total), amongst others. In the area of ML, things like AWS DeepComposer, Amazon SageMaker Studio, and Amazon Fraud Detector topped the list. While in the Compute, Analytics, and Networking space, Amazon EC2 Inf1 Instances, AWS Local Zones, AWS Outposts, Amazon Redshift Data lake, AWS Transit Gateway Network Manager, and Inter-Region Peering were at the forefront. Here at 2nd Watch we love the cutting-edge ML feature announcements like everyone else, but we always have our eye on those announcements that key-in on what our customers need now – announcements that can have an immediate benefit for our customers in their ongoing cloud journey.
All About the Network
In Matt Lehwess’ presentation, Advanced VPC design and new capabilities for Amazon VPC, he kicked off the discussion with a poignant note of, “Networking is the foundation of everything, it’s how you build things on AWS, you start with an Amazon VPC and build up from there. Networking is really what underpins everything we do in AWS. All the services rely on Networking.” This statement strikes a chord here at 2nd Watch as we have seen that sentiment in action. Over the last couple years, our customers have been accelerating the use of VPCs, and, as of 2018, Amazon VPCs is the number one AWS service used by our customers, with 100% of them using it. We look for that same trend to continue as 2019 comes to an end. It’s not the sexiest part of AWS, but networking provides the foundation that brings all of the other services together. So, focusing on newer and more efficient networking tools and architectures to get services to communicate is always at the top of the list when we look at new announcements. Here are our takes on these key announcements.
One exciting feature announcement in the networking space is Inter-Region Peering for AWS Transit Gateway. This feature allows the ability to establish peering connections between Transit Gateways in different AWS Regions. Previously, connectivity between two Transit Gateways could only be done through a Transit VPC which included the overhead of running your own networking devices as part of the Transit VPC. Inter-Region peering for AWS Transit Gateway enables you to remove the Transit VPC and connect Transit Gateways directly.
The solution uses a new static attachment type called a Transit Gateway Peering Attachment that, once created, requires an acceptance or rejection from the accepter Transit Gateway. In the future, AWS will likely allow dynamic attachments, so they advise you to create unique ASNs for each Transit Gateway for the easiest transition. The solution also uses encrypted VPC peering across the AWS backbone. Currently Transit Gateway inter-region peering support is available for gateways in US East (Virginia), US East (Ohio), US West (Oregon), EU (Ireland), and EU (Frankfurt) AWS Regions with support for other regions coming soon. You also can’t peer Transit Gateways in the same region.
(Source: Matt Lehwess: Advanced VPC design and new capabilities for Amazon VPC (NET305))
On the surface the ability to connect two Transit Gateways is just an incremental additional feature, but when you start to think of the different use cases as well as the follow-on announcement of Multi-Region Transit Gateway peering and Accelerated VPN solutions, the options for architecture really open up. This effectively enables you to create a private and highly-performant global network on top of the AWS backbone. Great stuff!
AWS Transit Gateway Network Manager
This new feature is used to centrally monitor your global network across AWS and on premises. The Transit Gateway network manager simplifies operational complexity of managing networks across regions and remote locations. This AWS feature is another to take a dashboard approach to provide a simpler overview of your resources that may be spread over several regions and accounts. To use it, you create a Global Network within the tool which is an object in the AWS Transit Gateway Network Manager service that represents your private global network in AWS. It includes your AWS Transit Gateway hubs, their attachments, and on-premises devices, sites, and links. Once the Global Network is created, you extend the configuration by adding in Transit Gateways, information about your on-premises devices, sites, links, and the Site-to-Site VPN connections with which they are associated, and start using it to visualize and monitor your network. It includes a nice geographic world map view to visualize VPNs (if they’re up/down impaired) or Transit Gateway Peering connections.
There’s also a nice Topology feature that shows VPCs, VPNs, Direct Connect gateways, and AWS Transit Gateway-AWS Transit Gateway peering for all registered Transit gateways. It provides an easier way to understand your entire global infrastructure from a single view.
Another key feature is the integration with SD-WAN providers like Cisco, Aviatrix, and others. Many of these solutions will integrate with AWS Transit Gateway Network Manager and automate the branch-cloud connectivity and provide end-to-end monitoring of the global network from a single dashboard. It’s something we look forward to exploring with these SD-WAN providers in the future.
AWS Local Zones
AWS Local Zones in an interesting new service that addresses challenges we’ve encountered with customers. Although listed under Compute and not Networking and Content Delivery on the re:Invent 2019 announcement list, Local Zones is a powerful new feature with networking at its core.
Latency tolerance for applications stacks running in a hybrid scenario (i.e. app servers in AWS, database on-prem) is a standard conversation when planning a migration. Historically, those conversations would be predicated by their proximity to an AWS region. Depending on requirements, customers in Portland, Oregon may have the option to run a hybrid application stack, where those in Southern California may have been excluded. The announcement of Local Zones (initially just in Los Angeles) opens up those options to markets that were not previously available. I hope this is the first of many localized resource deployments.
That’s no Region…that’s a Local Zone
Local Zones are interesting in that they only have a subset of the services available in a standard region. Local Zones are organized as a child of a parent region, notably the Los Angeles Local Zone is a child of the Oregon Region. API communication is done through Oregon, and even the name of the LA Local Zone AZ maps to Oregon (Oregon AZ1= us-west-2a, Los Angeles AZ1 = us-west-2-lax-1a). Organizationally, it’s easiest to think of them as remote Availability Zones of existing regions.
As of December 2019, only a limited amount of services are available, including EC2, EBS, FSx, ALB, VPC and single-zone RDS. Pricing seems to be roughly 20% higher than in the parent region. Given that this is the first Local Zone, we don’t know whether this will always be true or if it depends on location. One would assume that Los Angeles would be a higher-cost location whether it was a Local Zone or full region.
All the Things
To see all of the things that were launched at re:Invent 2019 you can check out the re:Invent 2019 Announcement Page. For all AWS announcements, not just re:Invent 2019 launches (e.g. Things that launched just prior to re:Invent), check out the What’s New with AWS webpage. If you missed the show completely or just want to re-watch your favorite AWS presenters, you can see many of the re:Invent presentations on the AWS Events Youtube Channel. After you’ve done all that research and watched all those videos and are ready to get started, you can always reach out to us at 2nd Watch. We’d love to help!
AWS re:Invent always presents us with a cornucopia of new cloud capabilities to build with and be inspired by, so listing just a few of the top takeaways can be a real challenge.
There are the announcements that I would classify as “this is cool, I can’t wait to hack on this,” which for me, a MIDI-aficionado and ML-wannabe, would include DeepComposer. Then there are other announcements that fall in the “good to know in case I ever need it” bucket such as AWS LocalZones. And finally, there are those that jump out at us because “our clients have been asking for this, hallelujah!”
I’m going to prioritize this list based on the latter group to start, but check back in a few months because, if my DeepComposer synthpop track drops on SoundCloud, I might want to revisit these rankings.
“AWS Compute Optimizer uses machine learning techniques to analyze the history of resource consumption on your account and make well-articulated and actionable recommendations tailored to your resource usage.”
Our options for EC2 instance types continues to evolve and grow over time. These evolutions address optimizations for specialized workloads (e.g., the new Inf1 instances), which means better performance-to-cost for those types of workloads.
The challenge for 2nd Watch clients (and everyone else in the Cloud) is maintaining an up-to-date knowledge of the options available and continually applying the best instance types to the needs of their workloads on an ongoing basis. That is a lot of information to keep up on, understand, and manage, and you’re probably wondering, “how do other companies deal with this?”
The ones managing it best have tools (such as CloudHealth) to help, but cost optimization is an area that requires continual attention and experience to yield the best results. Where AWS Compute Optimizer will immediately add value is surfacing inefficiencies at zero cost of 3rd party tools to get started. You will need to have the CloudWatch agent installed to gather OS-level metrics for the best results, but this is a standard requirement for these types of tools. What remains to be seen in the coming months is how Compute Optimizer compares to the commercial 3rd party tools on the market in terms of uncovering overall savings opportunities. However, the obvious advantage for 3rd party tools remaining unaffected by this change will be their ability to optimize across multiple cloud service providers.
“Amazon Elastic Container Service (ECS) now supports Windows group Managed Service Account (gMSA), a new capability that allows ECS customers to authenticate and authorize their Windows containers with network resources using an Active Directory (AD). Customers can now easily use Integrated Windows Authentication with their Windows containers on ECS to secure services.”
This announcement was not part of any keynote, but thanks to fellow 2nd Watcher and Principal Cloud Consultant, Joey Yore, bringing it to my attention, it is definitely making my list. Over the course of the past year, several of our clients on a container adoption path for their .NET workloads were stymied by this very lack of Windows gMSA support.
Drivers for migrating these .NET apps from EC2 to containers includes easier blue/green deployments for faster time-to-market, simplified operations by minimizing overall Windows footprint to monitor and manage, and cost savings also associated with the consolidated Windows estate. The challenge encountered was with the authentication for these Windows apps, as without the gMSA feature, the applications would require a time-intensive refactor or leverage an EC2 based solution with management overhead. This raised questions about the commitment of AWS to Windows containers in the long term, and thankfully, with this release, it signals that Windows is not being sidelined.
#3 AWS Security Hub Gets Smarter
Third on the list is a 2-for-1 special because security and compliance is one of the most common areas our clients have come to us for help. Cloud gives builders all of the tools they need to build and run secure applications, but defining controls and ensuring their continual enforcement requires consistent and deliberate work. In response to this need we’ve seen AWS releasing more services that streamline activities for security operations teams. In that list of tools are Amazon GuardDuty, Amazon Macie, and, more recently, AWS Security Hub, which these two selections integrate with:
“AWS IAM Access Analyzer generates comprehensive findings that identify resources that can be accessed from outside an AWS account. AWS IAM Access Analyzer does this by evaluating resource policies using mathematical logic and inference to determine the possible access paths allowed by the policies. AWS IAM Access Analyzer continuously monitors for new or updated policies, and it analyzes permissions granted using policies for their Amazon S3 buckets, AWS KMS keys, Amazon SQS queues, AWS IAM roles, and AWS Lambda functions.”
If you’ve worked with IAM, you know that without deliberate design and planning, it can become an unwieldy mess quickly. Disorganization with your IAM policies means you run the risk of creating inadvertent security holes in your infrastructure, which might not be immediately apparent. This new feature to AWS Security Hub streamlines the process for surfacing those latent IAM issues that may have otherwise gone unnoticed.
“Amazon Detective is a new service in Preview that makes it easy to analyze, investigate, and quickly identify the root cause of potential security issues or suspicious activities. Amazon Detective automatically collects log data from your AWS resources and uses machine learning, statistical analysis, and graph theory to build a linked set of data that enables you to easily conduct faster and more efficient security investigations.”
The result of Amazon’s acquisition of Sqrrl in 2018, Amazon Detective is another handy tool that helps separate the signal from the noise in the cacophony of cloud event data generated across accounts. What’s different about this service as compared to others like GuardDuty is that it builds relationship graphs which can be used to rapidly identify links (edges) between events (nodes). This is a powerful capability to have when investigating security events and the possible impact across your Cloud portfolio.
“EC2 Image Builder is a service that makes it easier and faster to build and maintain secure images. Image Builder simplifies the creation, patching, testing, distribution, and sharing of Linux or Windows Server images.”
2nd Watch clients have needed an automated solution to “bake” consistent machine images for years, and our “Machine Image Factory” solution accelerator was developed to efficiently address the need using tools such as Hashicorp Packer, AWS CodeBuild, and AWS CodePipeline.
The reason this solution has been so popular is that by having your own library of images customized to your organizations requirements (eg, security configurations, operations tooling, patching), you can release applications faster, with greater consistency, and without burdening your teams’ time or focus watching installation progress bars when they can be working on higher business value activities.
What’s great about AWS releasing this capability as a native service offering is that it is making a best-practice pattern even more accessible to organizations without confusing the business outcome with an array of underlying tools being brought together to make it happen. If your team wants to get started with EC2 Image Builder but you need help with understanding how to get from your current “hand crafted” images to Image Builder’s recipes and tests, we can help!
“AWS Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to virtually any datacenter, co-location space, or on-premises facility for a truly consistent hybrid experience. AWS Outposts is ideal for workloads that need low latency access to on-premises applications or systems, local data processing, and to securely store sensitive customer data that needs to remain anywhere there is no AWS region, including inside company-controlled environments or countries.”
It’s 2019, and plants are now meat and AWS is hardware you can install in your datacenter. I will leave it to you to guess which topic has been more hotly debated on the 2nd Watch Slack, but amongst our clients, Outposts has made its way into many conversations since its announcement at re:Invent 2018. Coming out of last week’s announcement of Outposts GA, I think we will be seeing a lot more of this service in 2020.
One of the reasons I hear clients inquiring about Outposts is that it fills a gap for workloads with proximity or latency requirements to manufacturing plants or another type of strategic regional facility. This “hyper-local” need echoes the announcement for AWS Local Zones, which presents a footprint for AWS cloud resources targeting a specific geography (Los Angeles, CA initially).
Of course, regional datacenters and other hyperconverged platforms exist to run these types of workloads already, but what is so powerful about Outposts is that it brings the Cloud operations model back to the datacenter and the same cloud skills that your teams have developed and hired for don’t need to be stunted to learn a disparate set of skills on a niche hardware vendor platform that could be irrelevant 3 years from now.
I’m excited to see how these picks and all of the new services announced play out over the next year. There is a lot here for businesses to implement in their environments to drive down costs, improve visibility and security, and dial in performance for their differentiating workloads.
Head over to our Twitter account, @2ndWatch, if you think there should be others included in our top 5 list. We’d love to get your take!
With a week full of sessions, bootcamps and extra-curriculars at AWS re:Invent, you might not have had time to make it to our breakout session.
Watch ‘The Simple Path to AMS’ On-Demand
Learn how to accelerate your journey to the cloud by using AWS Managed Services (AMS), including the process for assessing, migrating and operationalizing your infrastructure from your on-premise datacenter or existing cloud environment to AMS. Discover key steps to streamline this process using automation and infrastructure as code to set up network connectivity, access management, logging, monitoring, backups and configuration as well as integration points for an existing managed service provider to seamlessly work with AMS.
Thursday marked the last full day of AWS re:Invent 2019 and the morning after another outstanding 2nd Watch party. If you attended, it is understandable if you were unable to make Werner Vogels’ keynote address. Have no fear, 2nd Watch’s Victoria Geronimo has recapped all the highlights for you in her blog post, or you can watch it here. This year, Vogels focused more on how AWS builds to support microservices instead of on new announcements. As usual, his t-shirt choice was a huge topic of conversation.
It has been another great week here in Vegas, and again I am amazed at all the new and interesting people we get to talk to during this conference. It is truly a global experience getting to talk to people from all over the world and some AWS Heroes. I hope we got a chance to meet you at the 2nd Watch booth. If you needed some relaxation time, AWS provided plenty of areas and opportunities to play including Broomball, Dodgeball and the final party, re:Play, which featured Anderson Paak, as well as A-Trak, Jamestown Revival, Jen Lasher, Miya Folick, and STS9.
A few of the interesting announcements on Thursday included:
Amazon Fraud Detector, a fully managed service that makes it easy to identify potentially fraudulent online activities such as online payment fraud and the creation of fake accounts
UltraWarm, a fully managed, low-cost, warm storage tier for Amazon Elasticsearch Service that takes a new approach to providing hot-warm tiering in Amazon Elasticsearch Service, offering up to 900TB of storage at almost a 90% cost reduction over existing options
Advanced Query Accelerator (AQUA) for Amazon Redshift is a new distributed and hardware-accelerated cache that enables Redshift to run up to 10x faster than any other cloud data warehouse
As usual, the announcements this week show that AWS continues to listen to its customers and release services to fill those needs. There are still sessions going on today and thousands heading to the airport. Travel safe and see everyone next year November 30 – December 4, 2020 in Las Vegas.
AWS re:Invent kicked off Day 4 with its third and final keynote speech, delivered by Amazon.com’s CTO, Dr. Werner Vogels. 2nd Watch was up early with the birds and excited to hear about the architectural investments AWS is making this year.
The name of the game of this keynote was micro. From Nitro to Fargate to use cases at Vanguard, AWS had microservices up on its moodboard because it allows for more efficient services, greater customization, and speedier deployments.
Vogels began the keynote on Nitro System, AWS’ innovative virtualization platform that will be supporting core services. Hypervisors are integral to the cloud and offer multiple functionalities – protecting hardware, virtualizing storage and CPU, etc. Vogels described them as “monolithic,” which translates to fewer customization options. Inspired by microservices, Nitro modularizes each of these functionalities and offloads them onto hardware and software. By breaking each of these capabilities apart, we can tailor each and build a more efficient and less costly platform for our AWS services. Nitro is at the heart of many of AWS’ newer initiatives, including Outposts, VMware, Bare Metal, and Live Updates.
Vogels also gifted us security nerds with good news – Nitro encrypts everything out of the box and passes on Domain 0 (aka Dom0). Dom0 is a critical functionality of Xen hypervisors and has special privileges allowing it to access hardware. This means that if someone can exploit Dom0, they have direct access to the hardware. In fact, Nitro prohibits admin access entirely.
Clare Liguori, Principal Software Engineer for AWS, took the stage next to reinforce re:Invent’s earlier announcement that Fargate will be supporting EKS. With a clever trivia game demonstration, Clare showed how Fargate allows for better capacity and efficiently isolates containers. She also spent time on Firecracker – a lightweight micro-VMs for containers and serverless applications. She even threw the repo on screen for us! https://github.com/firecracker-microvm/firecracker.
Next, Jeff Dowds, IT Executive from Vanguard, presented a compelling story of how Vanguard developed a Cloud Construction Team to adopt a serverless environment, quickly get a cloud initiative in practice, and reduce overall computing costs.
Reiterating the micro theme, Vogels came back on to discuss the theory behind their cell-based architecture, which enables evolvable architecture and reduces blast radius. Things fail all the time, and if it’s a cell rather than a monolith, it’s harder to bring down the whole architecture. Vogels then discussed Amazon Builder’s Library, which empowers users with write ups on how Amazon architects its own resources in AWS – truly a great resource for anyone building on AWS.
Next up, Sebastian de Halleax, COO of Saildrone, presented how they use AWS to innovate in new ways on the unexplored territory of oceans (wow!). Their Ocean Surveyor is mapping the sea floor and sending data in realtime using AWS. They’ll use this innovation to see how the sea affects humans and vice versa. It’s pretty great to see how AWS is leveraged to solve real world issues outside of the IT realm.
Next up was Dr. Martin Hofmann, Group CIO of Volkswagen AG, which is leveraging AWS Autonomous Factories to innovate their supply chain management. Vogels noted how most equipment is old and outdated at manufacturing plants. Autonomous Factories – used at AWS warehouse and at Volkswagen – envision automating not only manufacturing but repairing of machines. Think Wall-E!