Network World’s Brandon Butler gives a tour of the Amazon Web Services re:Invent expo floor, and 2nd Watch booth, where companies are showing off products and services that support the AWS cloud.
Covanta Energy and 2nd Watch talk with SiliconANGLE Media at AWS re:Invent 2016. Find out why Covanta decided to go all-in on Amazon Web Services and how 2nd Watch helped them make the transition in only 16 weeks.
Every enterprise knows by now that it can save money by simply lifting and shifting workloads to the cloud, but many are missing the larger opportunity to also make money by moving. While quick costs savings are good for the bottom line, they do little to move the top line numbers. To achieve both savings and earnings, corporate thinking about technologies must change in order to enable faster processes leveraged enterprise-wide.
In this AWS re:Invent 2016 breakout session we explored multiple customer success stories where the customers have evolved from leveraging basic compute and storage products (EC2 and S3) to integrating new services into operations by leveraging Lambda, DynamoDB, CodeDeploy, etc. Once this is achieved, enterprises are enabled to manage and deploy code rapidly in a programmatic and elastic secure network, ensuring governance and security standards across the globe. We also looked at the migration process trusted by hundreds of clients as well as how to cope with the process and people components that are so important to enable agility, while focusing heavily on the technology.
Dive deep into the technology that allows the world’s largest beverage manufacturer to manage hundreds of AWS Accounts, hundreds of workloads, thousands of instances, and hundreds of business partners around the globe. The company’s Configuration Management System has Puppet at the core and relies on over a dozen core and emerging AWS products across accounts, availability zones and regions. This complex and globally-available system ensures all of company’s workloads in AWS meet corporate policies but also allows for rapid scale of both consumer and enterprise workloads.
After the deluge of announcements during Andy Jassy’s Wednesday keynote, I had a hard time imagining what else AWS could possibly have left to announce yesterday. Of course, in typical AWS fashion, they had a lot more to talk about.
Werner Vogel, Amazon’s CTO, laid out a strong case that developers, data analysts, and basically just about anyone involved with data spend 80 percent of their time preparing data to become usable — and not always successfully — while only spending 20% of their time on the actually analysis and use of that information. He laid out a vision for a new, modern data architecture – one that would flip that equation and therefore transform the way with which we can generate value and insight from all our data sources. The transformer theme, woven throughout Vogel’s talk (and even his shirt), was so pronounced that one wonders why the new AWS Snowmobile didn’t have an Optimus Prime paint job.
In pursuit of enabling this transformation, the Keynote showcased a large number of new and enhanced services, most centered around removing a lot of the scut work that reduces development speed, efficiency, and agility.
AWS OpsWorks for Chef: A fully managed Chef Automation environment that helps take a lot of the work out of continuous deployment.
Amazon EC2 Systems Manager: A suite of tools for task automation, package installation, resource configuration, and patching on Amazon EC2.
AWS Codebuild: A fully managed and extensible build service for compiling source code and running unit s. Codebuild integrates with a wide array of AWS services, and helps make CI/CD pipelines more efficient.
Personal Health Dashboard: Allowing developers to gain visibility into service health issues that may be affecting their application.
Amazon X-Ray: Another debuting tool, X-Ray lets developers analyze, visualize and debug distributed applications and identify performance bottlenecks.
AWS Shield: A new AWS security tool designed to provide layer 3/4 DDoS protection to web applications.
AWS Batch: Batch offers fully-managed, highly scalable batch processing without having to install batch processing software, manage servers, or worry about the finer points of job scheduling.
AWS Glue : A fully-managed data catalog and ETL service that makes it easy to transfer data among data stores while also simplifying associated tasks such as data discovery, conversion, and mapping.
Amazon Pinpoint: A new analytics tool to improve the behavioral analysis and engagement with mobile customers.
AWS Step Functions: Lets developers organize the components of distributed applications using visual workflows. Users can step through functions at scale, improving debugging.
Blox: A collection of open source projects for container management and orchestration.
Lambda@Edge: While just a preview at this point, this service enables Lambda functions at AWS edge locations, as well as execution in response to CloudFront events.
Sessions and Events
As usual, the conference agenda was stuffed to the gills with interesting and useful events, session, and demos. The meat of AWS, of course, is the wide array of breakout sessions. I was able to attend a few, three of which were particularly strong.
Lee Atchison from New Relic gave a particularly interesting talk on cloud monitoring and how best to architect applications and infrastructure to ensure they are fully measurable. This is challenging under any circumstances, but particularly so as applications and their underlying resources become more dynamic, more ephemeral. Gannet News discussed how they were able to transform their systems in this regard, offering a lot of insight into how to create and manage dynamic IT.
More architectural advice, this time with a focus on security, came from an AWS-led session on architecting end-to-end security in the enterprise, while a related session, also AWS-led, explored how to best automate security event response in such an architecture. This last is critical to have scalability and predictability when remediating security issues. In combination, these two sessions helped lay out an approach for highly dynamic, manageable, and secure systems.
The expo hall was very heavily-trafficked, with booths offering monitoring solutions and security analysis and governance being especially busy. IoT was also a topic of great interest during the conference, with several intriguing vendor offerings.
Check back Monday for our full conference recap.
-John Lawler, Sr. Product Manager
It’s all about The Transformation
At this morning’s AWS re:Invent keynote, AWS shared quite a mountain of information, and a toolbox of new services, all based around helping companies change their businesses and the way they look at technology. Transformation was the keyword for this presentation, and it was apparent in the tools and tone taken throughout the whole two and a half hours. The focus was on providing the tools to the “Transformers” (Highlighted by Vogel’s Autobot T-shirt), and enabling them to do amazing things for their customers. Vogel’s keynote was less about infrastructure, more about the software and how to get it into the hands of your customers, and how the toolbox that AWS continues to expand can help. It’s not entirely about AWS though… it starts with their customers.
AWS: To Be the Most Customer Centric IT Company on Earth
There’s a large drive from all the teams at AWS to focus on the needs of their customers (that’s you by the way). In fact, this couldn’t be more evident than with their new offering called AWS Blox, an open source scheduler for ECS that’ll be co-developed with the community. This can also be seen in their 5 customer centric objectives:
- Protect the customers at all times.
- Listen closely to customers and act.
- Give customers choice.
- Work backwards from the customer.
- Help customers transform.
This led nicely into Jeff Lawson’s (CEO / Chairman – Twilio) presentation which revolved around software development. The two things to take away from this were a couple of quotes: 1. “Building software is a mindset, not a skillset,” which speaks immeasurably to the idea of the enveloping purpose of software in the first place. Software drives products to customers. And 2. “Companies that win are companies that ship software.”
How can we help you be a Transformer?
There are a plethora of modern day processes revolving around Agile practices, which involve feature deployment speed to your customers. The big, main point here is that Amazon really wants to take as much of the waste off of their customers’ shoulders as possible and manage it for them. This is one of the fundamental principals in lean manufacturing and Agile development processes. Cut waste, so your people can concentrate on what’s important to your customer – Providing stellar products and features.
To that end, AWS already provides everything you’ll need as far as infrastructure is concerned. Need a thousand instances for a load ? Spin them up, run your , then tear them down, and only pay for that hour you had them up. That’s the bread and butter. Where AWS is moving now is to help that development pipeline and to provide the tools to do it.
First and foremost, they’ve updated their Well Architected Framework (along with all the underlying documentation) to include a 5th pillar:
- Performance Efficiency
- Cost Optimization
- Operational Excellence (This is where Automation and CI/CD pipelines come into play.)
Transforming Operational Excellence
Automation is the name of the game here. The existing tools have gotten some updates, and there are some new ones to add to your armory as well.
AWS CloudFormation has seen a ton of updates this past year including role-based stack creation, failure recovery, resource schemas and last but by far not least, yaml support! Configuration management (in the form of Chef) has gotten a BIG boost in their new AWS Opsworks For Chef Automate, a fully managed chef server. Oh, and managing system level patching and resource configuration? They’ve got that covered as well with the Amazon EC2 Systems Manager. The Biggest changes come to help your CI/CD pipeline. The new AWS CodeBuild will build and your projects and fills out the pipeline toolset (between CodeCommit and CodeDeploy). What about insight into your application? The fantastic looking X-Ray will allow insight into your applications on a very deep level, with a smart looking UI to boot. Another nice looking UI of a tool to handle managing events from your infrastructure is AWS Personal Health Dashboard. This tool will help you manage responses to your events, and can be tied into Lambda for automation.
Security is number one with AWS, so it’s no surprise that they’re offering two new tools to help protect against the common DDOS attack. The first, AWS Shield will help protect against some of the more common DDOS attack vectors. The best thing about it? Everyone gets it FOR FREE! You use AWS, you get AWS Shield. That simple. AWS Shield Advanced is for more complex attacks and is a paid service that you can opt in for if you feel the need.
Transforming your Data
Amazon’s cloud offering levels the playing field when it comes to resource procurement. Small companies can now compete with the big ones since they draw from the same pool and have the same tools available to them (regardless of size). So what’s your competitive differentiator? Data. That’s why another focus of this past year has been on Big Data.
AWS already has a lot going for it with data analytics, from ingestion tools like Kinesis and Snowball to processing with EMR, there just seemed to be one thing missing: AWS Glue. AWS Glue pulls together all the components of Modern Data Warehouses into a comprehensive architecture for data analytics. From data ingestion to data quality, source data preservation to orchestration and job scheduling, it looks like AWS Glue will manage it all. Also on the processing end, the new AWS Batch tool will manage batch processing at any scale.
Transforming your Application Architecture
Amazon now provides 3 different architectures and payment styles when it comes to application development (or deployment if you look at it that way) – Virtualization, which is already quite robust in their compute ecosystem; Containers, which have an ever maturing product in ECS; and Serverless, which is handled quite well through services like AWS Lambda. Virtualization didn’t get a particular mention here, but Containerization did. Blox was already mentioned above, but there was also a “coming soon” drop here as well. Looks like we’ll be seeing some kind of task placement engine in the near future.
Next up were new offerings around Lambda. The first, and one that will surely broaden the adoption of serverless architectures, is the inclusion of the C# language into the list of supported languages. To cut back on possible latency issues, you can now run Lambda functions at CloudFront locations using the new AWS Lambda@Edge. To help coordinate all the components of your distributed applications, you now have AWS Step Functions. This tool will allow you to coordinate all your bits and pieces using a visual workflow.
There’s a lot of potential for transforming your business here.
Like always, AWS doesn’t force you to use any particular tool or service, but they have a lot of what you need to develop products and features the right way. They’ve made some serious strides to pull as much of the wasted, non-customer centric work away from your teams, and give them back that time to push more value to your customers. Amazon doesn’t yet approach the organizational / process side of the equation, so that will still fall to the customer. Once you figure it out though, it looks like AWS is positioned, and will continue to position itself, to help you and your teams make that transformation a reality.
-Craig Monson, Sr Automation Architect
AWS re:Invent 2016 day 2 is in the books. Attendees took part in AWS and Partner-led technical sessions, bootcamps, and GameDay and had the opportunity to interact with a wide range of Consulting and Technology Partners at reinvent:Central (the Partner & Solutions Expo).
Things started off with the re:Invent Keynote by AWS CEO Andy Jassy, and per the usual, he did not disappoint. We learned about the Four Superpowers that AWS gives organizations… Supersonic Speed, X-Ray Vision, Immortality, and Flight. and we heard success stories around this from several major corporations. Speakers from GE, Motorola, Workday, FINRA and McDonalds joined the Keynote and told their success stories as they relate to AWS and these Four Superpowers. We also heard from Patrick Gelsinger, CEO of VMware, and Jassy about the new partnership between AWS and VMware. They discussed the alliance and the strategic and technical benefits that it will provide their customers. Gelsinger notably quipped: “Your CIOs capital budget for DR should be zero. DR should entirely be in the cloud.”
The Summit Keynote showcased a number of new and expanded services to the AWS lineup:
- New C5 EC2 Instance Family: Next Generation Compute Optimized Instances with Intel Skylake processors
- New I3 EC2 Instance Family: Next Generation I/O Intensive Instances
- New R4 EC2 Instance Family: Next Generation Memory Optimized Instances
- New F1 EC2 Instance Family: Compute instances with a Field Programmable Gate Arrays (FPGAs)
- Updates to the T2 EC2 Instance Family: Low-Cost Instances with Burstable Performance
- xlarge with 16GiB (2x more) memory and 2 vCPU
- 2xlarge with 32GiB Memory (4x more) and 2vCPU
- Addition of Elastic GPUs to all EC2 instance types
- PostgreSQL support for RDS Aurora
- Amazon Athena: Provides serverless interactive queries for data stored in S3 using Standard SQL
- Amazon Rekognition: Image recognition service that enables applications to easily recognize, search, and understand image details
- Amazon Polly: Text to speech service with 47 voices and 27 languages
- Amazon LEX: Natural language understanding and automatic speech recognition service (It’s what’s inside Alexa.)
Jassy unveiled the four new EC2 instance families – three of which are the la iterations on previous generations – listed above as well as the expansion of the T2 line. and types. The completely new EC2 instance family was the F1 class. F1 instances offer programmable hardware via an FPGA which allows for writing custom code that executes directly on the FPGA speeding up the processing time for discrete tasks. We also learned that AWS is adding Elastic GPUs as an option (handled much like an EBS volume) that will be available to attach to any EC2 instance to boost graphics processing performance.
Amazon Athena was announced today and should be a welcomed addition for anyone looking to turn their S3 data into a data-warehouse/lake instantaneously without incurring the costs associated with a Redshift cluster, infrastructure, or other data-warehousing platform. Athena provides serverless interactive queries for data stored in S3 using Standard SQL. This is a truly awesome new service for building a data-warehouse inexpensively.
Perhaps one of the biggest announcements of the day came as it was announced that AWS will be releasing PostgreSQL support for Aurora. This will definitely help Aurora continue to reshape the face of how enterprise class databases are deployed in the cloud. Speaking of Aurora reshaping database deployment in the cloud, Jassy mentioned that since its launch Aurora is the fasting-growing service AWS has ever released with 14k+ migrations to date.
The launch of the Amazon AI platform was another major announcement made yesterday. Leveraging the IP and tooling they have developed internally, AWS will be releasing three new services under this new AI platform aimed at providing AI services to the masses. The first is Amazon Rekognition and provides an image detection and recognition engine. The next is Amazon Polly, which provides a text-to-speech engine (in 47 languages with 24 voices). The last is Amazon LEX, which is a natural language understanding and automatic speech recognition engine, which operates at the core of the Amazon Alexa platform. This is just another great example of Amazon providing internal tools and services they have developed and bringing them to a global marketplace due to customer demand.
Some of these new features are in “preview” or “developer” access mode and haven’t been released for general availability yet. You can find more information about all of these services and more, on the AWS APN blog: https://aws.amazon.com/blogs/apn/. For a really great detailed breakdown of the Keynote check out our blog post from yesterday: https://www.2ndwatch.com/blog/8-things-we-learned-from-andy-jassys-2016-reinvent-keynote/
There was a flurry of activity on the re:Invent Central expo floor, and traffic to the 2nd Watch booth was no exception. New customers were met, partnerships were discussed and all kinds of cloud expertise was disseminated to the masses.
Sessions and Events
The Agenda was chock-full of interesting and useful events, session, and demos in addition to the Keynote Address. Events like the Alexa Skill Con, Certification Lounge provided an all-day option for honing skills and gaining AWS certification. Some, like the live demo “Creating an AWS Data Lake on S3,” highlighted the newly-announced AWS services and features. There were also several physical fun activities over at The Park, like Hangry Humans, Mini golf, Mechanical Bull Riding and a Batting cage for attendees to get outside for some fresh air and remain active.
After hours kicked off with the re:Invent 2016 Pub Crawl featuring a number of great restaurants and pubs. The night was finished off with a bang at the annual 2nd Watch re:Invent After Party.
Come see us today at Booth #825 for your free limited edition t-shirt and a chance to win a Segway!
-Ryan Kennedy, Principal Cloud Automation Architect