After the deluge of announcements during Andy Jassy’s Wednesday keynote, I had a hard time imagining what else AWS could possibly have left to announce yesterday. Of course, in typical AWS fashion, they had a lot more to talk about.
Werner Vogel, Amazon’s CTO, laid out a strong case that developers, data analysts, and basically just about anyone involved with data spend 80 percent of their time preparing data to become usable — and not always successfully — while only spending 20% of their time on the actually analysis and use of that information. He laid out a vision for a new, modern data architecture – one that would flip that equation and therefore transform the way with which we can generate value and insight from all our data sources. The transformer theme, woven throughout Vogel’s talk (and even his shirt), was so pronounced that one wonders why the new AWS Snowmobile didn’t have an Optimus Prime paint job.
In pursuit of enabling this transformation, the Keynote showcased a large number of new and enhanced services, most centered around removing a lot of the scut work that reduces development speed, efficiency, and agility.
AWS OpsWorks for Chef: A fully managed Chef Automation environment that helps take a lot of the work out of continuous deployment.
Amazon EC2 Systems Manager: A suite of tools for task automation, package installation, resource configuration, and patching on Amazon EC2.
AWS Codebuild: A fully managed and extensible build service for compiling source code and running unit s. Codebuild integrates with a wide array of AWS services, and helps make CI/CD pipelines more efficient.
Personal Health Dashboard: Allowing developers to gain visibility into service health issues that may be affecting their application.
Amazon X-Ray: Another debuting tool, X-Ray lets developers analyze, visualize and debug distributed applications and identify performance bottlenecks.
AWS Shield: A new AWS security tool designed to provide layer 3/4 DDoS protection to web applications.
AWS Batch: Batch offers fully-managed, highly scalable batch processing without having to install batch processing software, manage servers, or worry about the finer points of job scheduling.
AWS Glue : A fully-managed data catalog and ETL service that makes it easy to transfer data among data stores while also simplifying associated tasks such as data discovery, conversion, and mapping.
Amazon Pinpoint: A new analytics tool to improve the behavioral analysis and engagement with mobile customers.
AWS Step Functions: Lets developers organize the components of distributed applications using visual workflows. Users can step through functions at scale, improving debugging.
Blox: A collection of open source projects for container management and orchestration.
Lambda@Edge: While just a preview at this point, this service enables Lambda functions at AWS edge locations, as well as execution in response to CloudFront events.
Sessions and Events
As usual, the conference agenda was stuffed to the gills with interesting and useful events, session, and demos. The meat of AWS, of course, is the wide array of breakout sessions. I was able to attend a few, three of which were particularly strong.
Lee Atchison from New Relic gave a particularly interesting talk on cloud monitoring and how best to architect applications and infrastructure to ensure they are fully measurable. This is challenging under any circumstances, but particularly so as applications and their underlying resources become more dynamic, more ephemeral. Gannet News discussed how they were able to transform their systems in this regard, offering a lot of insight into how to create and manage dynamic IT.
More architectural advice, this time with a focus on security, came from an AWS-led session on architecting end-to-end security in the enterprise, while a related session, also AWS-led, explored how to best automate security event response in such an architecture. This last is critical to have scalability and predictability when remediating security issues. In combination, these two sessions helped lay out an approach for highly dynamic, manageable, and secure systems.
The expo hall was very heavily-trafficked, with booths offering monitoring solutions and security analysis and governance being especially busy. IoT was also a topic of great interest during the conference, with several intriguing vendor offerings.
What a difference a year can make at AWS. In Andy Jassy’s AWS re:Invent 2015 Keynote address, which lasted just shy of 2 hours, he gave us some pretty stunning financial growth statistics and detailed the seven freedoms AWS provides its customers including; the freedom to build unfettered, to get the real value from your data, to get data into or out of the cloud easily, freedom from bad database relationships, freedom to migrate, freedom to secure your cake and eat it to, and freedom to say yes.
This year, in a presentation that lasted almost two and a half hours, Andy updated us on key corporate and re:Invent statistics, discussed the “4 Superpowers” AWS endows upon their customers and, along the way, introduced many significant new products and features that are sure to change the game moving forward. Here are the key things you can take away from this presentation:
AWS continues its exponential growth and commitment to its customers
The participant headcount at re:Invent this year increased to over 32,000 (up from 19,000 in 2015), Live Stream viewership increased from 38,000 in 2015 to over 50,000 and breakout session increased from 248 to over 400 this year. In that same time AWS has grown its business from a mere $7B run rate to over $13B and is by far the fas growing large scale technology company in the world. According to Gartner, AWS has several times the compute capacity of the next 14 cloud providers combined. With their continued investment in new features and technologies it is safe to say this lead will remain well into the future.
IT Professionals Need Even More Compute Options
Agility and speed are two of the main drivers compelling companies to move to the cloud (along with cost savings of course). AWS provides the most comprehensive line of products and deep functionality which enables companies to leverage the cloud to quickly adopt the cloud and speed new products and services to market. Andy emphasized AWS core strategy of having more than one data center in each region to provide customers the resiliency they require and that they continue to launch new products and add new features to existing products in order to make the cloud even more valuable.
AWS has more compute products than anyone else in the market but that is not stopping them from expanding their offering. Today AWS announced the following four new EC2 instance families:
T2 Family: Low-Cost Instances with Burstable Performance
xlarge with 16GiB (2x more) memory and 2 vCPU
2xlarge with 32GiB Memory (4x more) and 2vCPU
New R4 Family: Next Generation Memory Optimized Instances
New I3 Family: Next Generation I/O Intensive Instances
3 million IOPs (9x more) 4.88GiB (2x more) 15 2TB MVMe SSD (2.3x more) and 64 vCPUs (2x more)
New C5 Family: Next Generation Compute Optimized Instances with Intel Skylake processors
72 vCPUs (2x more), Skylake processor (2x greater performance), 12Gbps to EBS (3X greater throughput) and 144GiB memory (2.4x more)
New F1 family (Developer Preview): Instances with access to a Field Programmable Gate Array (FPGA) which can be utilized to more quickly and efficiently execute code for application specific workloads like seismic data, genomics, and financial analysis to name just a few. With the new F1 instances users can now run custom logic on EC2 bringing custom hardware acceleration to anyone.
In addition to these new compute products AWS announced Elastic GPUs for EC2 in sizes from 1 GiB up to 8 GiBs to allow you to attach and use graphics GPUs with an EC2 instance in the same way as an EBS Volumes. Another great new compute product announced today was Lightsail, which is in effect a Virtual Private Sever (VPS) made easy. Lightsail truly revolutionizes VPS by making the deployment a simple three step process for as little as $5 per month. Simply select your image, select your size, and pick a name and AWS does the rest (Launch VM, Attach SSD Storage, Manage IAM, Create Security Groups, setup DNS and create a Static IP) in the background. Now setting up a VPS is about as simple as it can get.
To quote Andy, “We love Ourselves Some Compute”, and it really shows with these new products.
Companies Need Better Analytics, and AWS Can Deliver
The ability to understand your customers and business grows more critical every day. AWS has a broad set of products including Amazon EMR, Elasticsearch, Kinesis, Redshift, QuickSite and Machine Learning. To compliment this comprehensive line of products AWS announced the launch of Amazon Athena. Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. This enables companies to query data without having to move data out of S3 or set up any infrastructure. You get fast results and pay only for the queries you run.
Artificial Intelligence and Deep Learning are clearly advancing how computers are solving intuitive problems. The challenge with AI and Deep Learning has always been that it takes very skilled team to actually make use of these technologies. AWS is working to bring the power of AI and Deep Learning to all developers through the release of the Amazon AI Services: Amazon Rekognition, Amazon Polly, and Amazon LEX.
Amazon Rekognition is an image recognition service that enables applications to easily recognize, search, and understand image details.
Amazon Polly is a new text to speech service with 47 voices and 27 languages.
Amazon LEX (It’s what’s inside Alexa) is a natural language understanding and automatic speech recognition service powered by deep learning which enable programmers to build natural conversational interactions in voice and text.
It’s hard for Enterprises to Stay Relevant and Grow
Only 12 of the companies in the first Fortune 500 list are still there today. To remain relevant and continue growing, enterprises need to be able to quickly capitalize on new technologies. The ones that do not will most likely face extinction in the future. Andy highlighted Workday, GE, Motorola, FINRA, and McDonald as companies who have used the cloud to either build or transform their business. Workday is a $1.5B company built from the ground up in the cloud. GE has moved 9,000 applications to AWS. Motorola is reinventing themselves as a communications platform using AWS services. FINRA is a financial market surveillance company using big data analytics and elastic compute at a massive scale to surveil, analyze, and protect financial markets for the SEC. FINRA chose AWS over traditional data centers not only for the cost savings, tooling, and elastic capabilities they provide but also sited security that is better than traditional private data-centers as being a major factor. McDonalds POS system with two hundred thousand registers is now running on AWS. Each of these companies can now bring new products and services to market faster and can easily scale to meet future demands.
Companies Need to Move Away from Customer Hostile Databases
We have all heard or experienced horror stories of old world databases. With their proprietary designs, lock-in pricing strategies, and punitive licensing agreements these databases have been at best a challenge for many customers. AWS has built a strategy to make open databases easier, more reliable, and faster to use with offerings that include MySQL, PostgreSQL, MariaDB, and Aurora. Aurora provides the speed and availability of commercial databases with the cost-effectiveness of open source and has become their fas growing service ever (with 14k+ migrations to date). And as of today Aurora now includes PostgreSQL support. The combination of these products make it much easier to move away from old world databases.
Choosing between On Premise to Cloud is No Longer a Binary Decision
AWS is making it easier to deploy hybrid IT infrastructure. Six weeks ago they announced a partnership with VMware to enable companies to run the same VMware software on AWS that they run in their data centers. This allows companies to use their existing VMware tools and seamlessly migrate workloads to and from the cloud greatly increasing flexibility while using the tools they are familiar with.
IoT is About to Change in Dramatic Fashion
IoT devices typically have very limited resources on board. To process information, the data has to be transported back to an on-premise or cloud datacenter. This increases latency and bandwidth costs while exposing customers to potential connectivity disruptions. To address these issues, AWS announced Greengrass which embeds Lambda compute (and other AWS services) in connective devices to provide local compute, messaging, and data caching. Soon Greengrass will be available on all Intel and Qualcomm IoT chips as well as Canonical and Annapurnalabs software.
Your Gonna Need a Bigger Truck
Last year AWS launched Snowball a petabyte-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of the AWS cloud. This product has been a huge success and after listening to users, they are expanding the service. Today AWS announce AWS Snowball Edge providing 100Tb of storage, on-board computing and clustering capabilities, an S3 endpoint and Greengrass inside.
But what about exabyte-scale data?… Believe it or not, enterprises are now faced with moving exabytes of data from on-premise storage to AWS. Moving that much data to the cloud using a 10Gbps dedicated connection would take 26 years. What to do? Use the newly released AWS Snowmobile! Snowmobile is a dedicated semi truck tractor trailer (yes they even drove one on stage) for data transfer. This 45 foot long rugged container and truck can be connected via fiber directly to your datacenter to collect your data then transported to AWS for upload 100 petabytes at a time taking an exabyte migration from 26 years down to 6 months.
AWS is not only the clear leader in the cloud space, but they continue to listen to customers and adapt their product portfolio to meet the ever changing needs of the business world. With up to a thousand new products and features launched each year, you can be sure they will continue to adapt and remain the leader in this space for many decades to come.
AWS kicked off its 2016 re:Invent conference in Las Vegas yesterday. Attendees took part in AWS and Partner-led technical sessions, bootcamps and a GameDay, and had the opportunity to interact with a wide range of Consulting and Technology Partners in the Partner & Solutions Expo.
The conference began with The Global Partner keynote, including an overview and some imonials from a wide range of partners and highlighting how AWS has evolved its business and the business of its customers. Terry Wise, Vice President of Global Alliances, Ecosystem and Channels at AWS, was joined by a number of guests throughout the keynote, including James Hamilton, Vice President and Distinguished Engineer of AWS, Adam Japhet, Head of Technology Services Architecture & Design, as well as many others. A wide gamut of topics were covered from a few technology partners, but the overall AWS sentiment was, “Our customers are doing cool things” and “Partners are doing very innovating things on behalf of their customers.”
Yesterday’s evening activities included an event in which James Hamilton, Vice President and Distinguished Engineer at AWS, gave us a peek into what makes up AWS infrastructure. It was a very enlightening view into the world of highly-scalable, highly-resilient distributed systems and the concerns you have to think about when building the world’s best platform as a service.
Announcements came in droves from the Partner Keynote.
AWS Service Delivery Program, which is designed to help you showcase your firm’s proven expertise and customer success in different AWS services (and in which 2nd Watch achieved Partner status for Amazon Aurora, Redshift, CloudFront and DMS).
AWS Public Sector Program, which recognizes APN Partners with solutions and experience delivering government, education, and nonprofit customer missions globally.
AWS IoT Competency recognizes industry-leading AWS Consulting and Technology Partners that provide proven technology and/or implementation capabilities for a variety of use cases including (though not limited to) intelligent factories, smart cities, energy, automotive, transportation, and healthcare.
AWS Financial Services Competency, which helps customers identify and connect with industry-leading Consulting and Technology APN Partners with solutions for banking and payments, capital markets, and insurance (and which 2nd Watch has also earned).
AWS Partner Solutions Finder (PSF), a new web-based tool meant to help customers easily filter, find, and connect with APN Partners to meet specific business needs.
VMware Cloud on AWS Partner Program that will be launched in 2017. The program will provide support for APN Partners that help customers deploy and operate VMware workloads on AWS.
2nd Watch presented breakout session “Lift and Evolve: Saving Money in the Cloud is Easy, Making Money Takes Help” The expert lineup included:
Chris Resch, EVP, Cloud Solutions
Christopher Nolan, Director of Product
Ryan Kennedy, Principal Automation Architect
In this session we looked at the cloud migration process trusted by hundreds of clients, as well as how to cope with the process and people components that are so important to enable agility, while focusing heavily on the technology. The technology showcased enables management of hundreds of AWS Accounts, hundreds of workloads, thousands of instances, and hundreds of business partners around the globe. The configuration management system has Puppet at the core and relies on over a dozen core and emerging AWS products across accounts, availability zones and regions. This complex and globally-available system ensures workloads in AWS meet corporate policies but also allows for rapid scale of both consumer and enterprise workloads.
Attendees were able to select from a plethora of breakout sessions. I want to highlight a couple of the sessions of the day that I thought were truly spectacular:
The first, hosted by Sony, dealt with breaking the bandwidth barrier in highly-available, low latency applications. They went into detail about the price we pay for network and computational overhead when store state. Dustin Pham and Alexander Filipchik made a good explanation of soft state and their innovative use of sticky sessions on ELBs for use with building indexes in Lucene.
The second was hosted by Riot games. Sean Maloney of Riot gave an excellent overview of their journey to deploying containerized infrastructure using Terraform. Riot really wanted to give total ownership of the application to their developers and was uncompromising in making that happen in a logical, safe and well-architected manner that abides by some of their key tenets.
The turnout at this year’s conference has been absolutely amazing. At last count there were about 33,000 people in attendance, making this the biggest re:Invent yet. It also looks to be the most exciting, and things are just getting started. Check back tomorrow morning for Wednesday’s conference recap.