How to Federate Amazon Redshift Access with Azure Active Directory

Single sign-on (SSO) is a tool that solves fundamental problems, especially in midsize and large organizations with lots of users.

End users do not want to have to remember too many username and password combinations. IT administrators do not want to have to create and manage too many different login credentials across enterprise systems. It is a far more manageable and secure approach to federate access and authentication through a single identity provider (IdP).

As today’s enterprises rely on a wide range of cloud services and legacy systems, they have increasingly adopted SSO via an IdP as a best practice for IT management. All access and authentication essentially flows through the IdP wherever it is supported. Employees do not have to remember multiple usernames and passwords to access the tools they need to do their jobs. Just as importantly, IT teams prevent an administrative headache. They manage a single identity per user, which makes tasks like removing access when a person leaves the organization much simpler and less prone to error.

The same practice extends to AWS. As we see more customers migrate to the cloud platform, we hear a growing need for the ability to federate access to Amazon Redshift when they use it for their data warehouse needs.

Database administration used to be a more complex effort. Administrators had to figure out which groups a user belonged to, which objects a user or group were authorized to use, and other needs—in manual fashion. These user and group lists—and their permissions—were traditionally managed within the database itself, and there was often a lot of drift between the database and the company directory.

Amazon Redshift administrators face similar challenges if they opt to manage everything within Redshift itself. There is a better way, though. They can use an enterprise IdP to federate Redshift access, managing users and groups within the IdP and passing the credentials to Amazon Redshift at login.

We increasingly hear from our clients, “We use Azure Active Directory (AAD) for identity management—can we essentially bring it with us as our IdP to Amazon Redshift?”

They want to use AAD with Redshift the way they use it elsewhere, to manage their users and groups in a single place to reduce administrative complexity. With Redshift, specifically, they also want to be able to continue managing permissions for those groups in the data warehouse itself. The good news is you can do this and it can be very beneficial.

Without a solution like this, you would approach database administration in one of two alternative ways:

  1. You would provision and manage users using AWS Identity and Access Management (IAM). This means, however, you will have another identity provider to maintain—credentials, tokens, and the like—separate from an existing IdP like AAD.
  2. You would do all of this within Redshift itself, creating users (and their credentials) and groups and doing database-level management. But this creates similar challenges to legacy database management, and when you have thousands of users, it simply does not scale.

Our technical white paper outlines how to federate access to Amazon Redshift using Azure Active Directory as your IdP, passing user and group information through to the database at login.

Download the technical white paper

-Rob Whelan, Data & Analytics Practice Director

Facebooktwitterlinkedinmailrss

How to Federate Amazon Redshift Access with Okta

Single sign-on (SSO) is a tool that solves fundamental problems, especially in midsize and large organizations with lots of users.

End users do not want to have to remember too many username and password combinations. IT administrators do not want to have to create and manage too many different login credentials across enterprise systems. It is a far more manageable and secure approach to federate access and authentication through a single identity provider (IdP).

As today’s enterprises rely on a wide range of cloud services and legacy systems, they have increasingly adopted SSO via an IdP as a best practice for IT management. All access and authentication essentially flows through the IdP wherever it is supported. Employees do not have to remember multiple usernames and passwords to access the tools they need to do their jobs. Just as importantly, IT teams prevent an administrative headache: They manage a single identity per user, which makes tasks like removing access when a person leaves the organization much simpler and less prone to error.

The same practice extends to AWS. As we see more customers migrate to the cloud platform, we hear a growing need for the ability to federate access to Amazon Redshift when they use it for their data warehouse needs.

Database administration used to be a more complex effort. Administrators had to figure out which groups a user belonged to, which objects a user or group were authorized to use, and other needs—in manual fashion. These user and group lists—and their permissions—were traditionally managed within the database itself, and there was often a lot of drift between the database and the company directory.

Amazon Redshift administrators face similar challenges if they opt to manage everything within Redshift itself. There is a better way, though. They can use an enterprise IdP to federate Redshift access, managing users and groups within the IdP and passing the credentials to Amazon Redshift at login.

We increasingly hear from our clients, “We use Okta for identity management—can we essentially bring it with us as our IdP to Amazon Redshift?” They want to use Okta with Redshift the way they use it elsewhere, to manage their users and groups in a single place to reduce administrative complexity. With Redshift, specifically, they also want to be able to continue managing permissions for those groups in the data warehouse itself. The good news is you can do this and it can be very beneficial.

Without a solution like this, you would approach database administration in one of two alternative ways:

  1. You would provision and manage users using AWS Identity and Access Management (IAM). This means, however, you will have another identity provider to maintain—credentials, tokens, and the like—separate from an existing IdP like Okta.
  2. You would do all of this within Redshift itself, creating users (and their credentials) and groups and doing database-level management. But this creates similar challenges to legacy database management, and when you have thousands of users, it simply does not scale.

Our technical white paper covers how to federate access to Amazon Redshift using Okta as your IdP, passing user and group information through to the database at login. We outline the step-by-step process we follow when we implement this solution for 2nd Watch clients, including the modifications we found were necessary to ensure everything worked properly. We explain how to set up a trial account at Okta.com, build users and groups within the organization’s directory, and enable single sign-on (SSO) into Amazon redshift.

Download the technical white paper

-Rob Whelan, Data & Analytics Practice Director

Facebooktwitterlinkedinmailrss

Top Enterprise IT Trends for 2021

Between the global pandemic and the resulting economic upheaval, it’s fair to say many businesses spent 2020 in survival mode. Now, as we turn the page to 2021, we wonder what life will look like in this new normalcy. Whether it is employees working from home, the shift from brick and mortar to online sales and delivery, or the need to accelerate digital transformation efforts to remain competitive, 2021 will be a year of re-invention for most companies.

How might the new normal impact your company? Here are five of the top technology trends we predict will drive change in 2021:

1. The pace of cloud migration will accelerate

Most companies, by now, have started the journey to the public cloud or to a hybrid cloud environment. The events of 2020 have added fuel to the fire, creating an urgency to maximize cloud usage within companies that now understand that the speed, resilience, security and universal access provided by cloud services is vital to the success of the organization.

“By the end of 2021, based on lessons learned in the pandemic, most enterprises will put a mechanism in place to accelerate their shift to cloud-centric digital infrastructure and application services twice as fast as before the pandemic,” says Rick Villars, group vice president, worldwide research at IDC. “Spending on cloud services, the hardware and software underpinning cloud services, and professional and managed services opportunities around cloud services will surpass $1 trillion in 2024,”

The progression for most companies will be to ensure customer-facing applications take priority. In the next phase of cloud migration, back-end functionality embodied in ERP-type applications will move to the cloud. The easiest and fastest way to move applications to the cloud is the simple lift-and-shift, where applications remain essentially unchanged. Companies looking to improve and optimize business processes, though, will most likely refactor, containerize, or completely re-write applications. They will turn to “cloud native” approaches to their applications.

2. Artificial intelligence (AI) and machine learning (ML) will deliver business insight

Faced with the need to boost revenue, cut waste, and squeeze out more profits during a period of economic and competitive upheaval, companies will continue turning to AI and machine learning to extract business insight from the vast trove of data most collect routinely, but don’t always take advantage of.

According to a recent PwC survey of more than 1,000 executives, 25% of companies reported widespread adoption of AI in 2020, up from 18% in 2019. Another 54% are moving quickly toward AI. Either they have started implementing limited use cases or they are in the proof-of-concept phase and are looking to scale up. Companies report the deployment of AI is proving to be an effective response to the challenges posed by the pandemic.

Ramping up AI and ML capabilities in-house can be a daunting task, but the major hyperscale cloud providers have platforms that enable companies to perform AI and ML in the cloud. Examples include Amazon’s SageMaker, Microsoft’s Azure AI and Google’s Cloud AI.

Edge computing will take on greater importance

For companies that can’t move to the cloud because of regulatory or data security concerns, edge computing is emerging as an attractive option. With edge computing, data processing is performed where the data is generated, which reduces latency and provides actionable intelligence in real time. Common use cases include manufacturing facilities, utilities, transportation, oil and gas, healthcare, retail and hospitality.

The global edge computing market is expected to reach $43.4 billion by 2027, fueled by an annual growth rate of nearly 40%, according to a report from Grand View Research.

The underpinning of edge computing is IoT, the instrumentation of devices (everything from autonomous vehicles to machines on the factory floor to a coffee machine in a fast-food restaurant) and the connectivity between the IoT sensor and the analytics platform. IoT platforms generate a vast amount of real-time data, which must be processed at the edge because it would too expensive and impractical to transmit that data to the cloud.

Cloud services providers recognize this reality and are now bringing forth specific managed service offerings for edge computing scenarios, such as Amazon’s new IoT Greengrass service that extends cloud capabilities to local devices, or Microsoft’s Azure IoT Edge.

4. Platform-as-a-Service will take on added urgency

To increase the speed of business, companies are shifting to cloud platforms for application development, rather than developing apps in-house. PaaS offers a variety of benefits, including the ability to take advantage of serverless computing delivering scalability, flexibility and quicker time to develop and release new apps. Popular serverless platforms include Amazon Lambda and Microsoft’s Azure Functions.

5. IT Automation will increase

Automating processes across the entire organization is a key trend for 2021, with companies prioritizing and allocating money for this effort. Automation can cut costs and increase efficiency in a variety of areas – everything from Robotics Process Automation (RPA) to automate low-level business processes, to the automation of security procedures such as anomaly detection or incident response, to automating software development functions with new DevOps tools.

Gartner predicts that, through 2024, enhancements in analytics and automatic remediation capabilities will refocus 30% of IT operations efforts from support to continuous engineering. And by 2023, 40% of product and platform teams will use AIOps for automated change risk analysis in DevOps pipelines, reducing unplanned downtime by 20%.

Tying it all together

These trends are not occurring in isolation.  They’re all part of the larger digital transformation effort that is occurring as companies pursue a multi-cloud strategy encompassing public cloud, private cloud and edge environments. Regardless of where the applications live or where the processing takes place, organizations are seeking ways to use AI and machine learning to optimize processes, conduct predictive maintenance and gain critical business insight as they try to rebound from the events of 2020 and re-invent themselves for 2021 and beyond.

Where will 2021 take you? Contact us for guidance on how you can take hold of these technology trends to maximize your business results and reach new goals.

-Mir Ali, Field CTO

Facebooktwitterlinkedinmailrss

Demystifying DevOps

My second week at 2nd Watch, it happened.  I was minding my own business, working to build a framework for our products and how they’d be released when suddenly, an email dinged into my inbox. I opened it and gasped. It read in sum:

  • We want to standardize on our DevOps tooling
  • We have this right now https://landscape.cncf.io/images/landscape.png
  • We need to pick one from each bucket
  • We want to do this next week with everyone in the room, so we don’t create silos

This person sent me the CNCF Landscape and asked us to choose which tool to use from each area to enable “DevOps” in their org. Full stop.

I spent the better part of the day attempting to respond to the message without condescension or simply saying ‘no.’ I mean, the potential client had a point. He said they had a “planet of tools” and wanted to narrow it down to 1 for each area. A noble cause. They wanted a single resource from 2nd Watch who had used all the tools to help them make a decision. They had very honorable intentions while they were clearly misguided from the start.

The thing is, this situation hasn’t just come up once in my career in the DevOps space. It’s a rampant misunderstanding of the part tooling, automation, and standardization play into the DevOps transformation. Normalizing the tech stack across your enterprise wastes time and creates enemies. This is not DevOps.

In this blog, I’ll attempt to share my views on how to approach your transformation. Hint: We’re not spending time in the CNCF Landscape.

Let’s start with a definition.

DevOps is a set of cultural values and organizational practices that improve business outcomes by increasing collaboration and feedback between Business Stakeholders, Development, QA, IT Operations, and Security.

A true DevOps transformation includes an evolution of your company culture, automation and tooling, processes, collaboration, measurement systems, and organizational structure.

DevOps does not have an ‘end state’ or level of maturity. The goal is to create a culture of continuous improvement.

DevOps methods were initially formed to bridge the gap between Development and Operations so that teams could increase speed to delivery as well as quality of product at the same time. So, it does seem fair that many feel a tool can bridge this gap. However, without increasing communication and shaping company culture to enable reflection and improvement, tooling ends up being a one-time fix for a consistently changing and evolving challenge. Imagine trying to patch a hole in a boat with the best patch available while the rest of the crew is poking holes in the hull. That’s a good way to imagine how implementing tooling to fix cultural problems works.

Simply stated – No, you cannot implement DevOps by standardizing on tooling.

Why is DevOps so difficult?

Anytime you have competing priorities between departments you end up with conflict and difficulty. Development is focused on speed and new features while Operations is required to keep the systems up and running. The two teams have different goals, some of which undo each other.

For example: As the Dev team is trying to get more features to market faster, the Ops team is often causing a roadblock so that they can keep to their Service Level Agreement (SLA). An exciting new feature delivered by Dev may be unstable, causing downtime in Ops. When there isn’t a shared goal of uptime and availability, both teams suffer.

CHART: Dev vs. Ops

Breaking Down Walls

Many equate the goal of conflict between Dev and Ops to a wall. It’s a barrier between the two teams, and oftentimes Dev, in their effort to increase velocity, will throw the new feature or application over the proverbial wall to Ops and expect them to deal with running the app at scale.

Companies attempt to fix this problem by increasing communications between the two teams, though, without shared responsibility and goals, the “fast to market” Dev team and the “keep the system up and running” Ops team are still at odds.

CI/CD ≠ DevOps

I’ll admit, technology can enable your DevOps transformation, though don’t be fooled by the marketing that suggests simply implementing continuous integration and continuous delivery (CI/CD) tooling will kick-start your DevOps journey. Experience shows us that implementing automation without changing culture just results in speeding up our time to failure.

CI/CD can enable better communication, automate the toil or rework that exists, and make your developers and ops team happier. Though, it takes time and effort to train them on the tooling, evaluate processes and match or change your companies’ policies, and integrate security. One big area that is often overlooked is executive leadership acceptance and support of these changes.

The Role of Leadership in DevOps

The changes implemented during a DevOps Transformation all have a quantifiable impact on a company’s profit, productivity and market share. It is no wonder that organizational leaders and executives have impact on these changes. The challenge is their impact is often overlooked up front in the process causing a transformation to stall considerably and producing mixed results.

According to the book Accelerate: Building and Scaling High Performing Technology Organizations by Nicole Forsgren, PhD, Jez Humble, and Gene Kim, the “characteristics of transformational leadership are highly correlated with software delivery performance.” Leaders need to up their game by enabling cross-functional collaboration, building a climate of learning, and supporting effective use and choice of tools (not choosing for the team like we saw at the beginning of this blog).

Leaders should be asking themselves upfront “how can I create a culture of learning and where do we start?” The answer to this question is different for every organization, though some similarities exist. One of the best ways to figure out where to start is to figure out where you are right now.

2nd Watch offers a 2-week DevOps Assessment and Strategy engagement that helps you identify and assess a single value stream to pinpoint the areas for growth and transformation. This is a good start to identify key areas for improvement. This assessment follows the CALMS framework originally developed by Damon Edwards and John Willis at DevOpsDays 2010 and then enhanced by Jez Humble. It’s an acronym representing the fundamental elements of DevOps:  Culture, Automation, Lean, Metrics, and Sharing.

  • Culture – a culture of shared responsibility, owned by everyone
  • Automation – eliminating toil, make automation continuous
  • Lean – visualize work in progress, limit batch sizes and manage queue lengths
  • Metrics – collect data on everything and provide visibility into systems and data
  • Sharing – encourage ongoing communication, build channels to enable this

While looking at each of these pillars we overlay the people, process and technology aspects to ensure full coverage of the methodology.

  • People:We assess your current team organizational structure, culture, and the role security plays. We also assess what training may be required to get your team off on the right foot. ​
  • Process:We review your collaboration methods, processes and best practices. ​
  • Technology:We identify any gaps in your current tool pipeline including image creation, CI/CD, monitoring, alerting, log aggregation, security and compliance and configuration management. ​

What’s Next?

DevOps is over 10 years old and it’s moving out of the phase of concept and ‘nice to have’ into necessity for companies to keep up in the digital economy. New technology like serverless, containers, and machine learning and a focus on security is shifting the landscape, making it more essential that companies adopt the growth mindset expected in a DevOps culture.

Don’t be left behind. DevOps is no longer a fad diet, it’s a lifestyle change. And if you can’t get enough of Demystifying Devops, check out our webinar on ‘Demystifying Devops: Identifying Your Path to Faster Releases and Higher Quality’ on-demand.

-Stefana Muller, Sr Product Manager

Facebooktwitterlinkedinmailrss

Troubleshooting VMware HCX

I was on a project recently where we had to set up VMware HCX in an environment to connect the on-premises datacenter to VMware Cloud on AWS for a Proof of Concept migration.  The workloads were varied, ranging from 100 MB to 5TB in size.  The customer wanted to stretch two L2 subnets and have the ability to migrate slowly, in waves.  During the POC, we found problems with latency between the two stretched networks and decided that the best course of action would be to NOT stretch the networks and instead, do an all-at-once migration.

While setting up this POC, I had occasion to do some troubleshooting on HCX due to connectivity issues.  I’m going to walk through some of the troubleshooting I needed to do.

The first thing we did was enable SSH on the NSX manager.  To perform this action, you go into the HCX manager appliance GUI and under Appliance Summary, start the SSH service.  Once SSH is enabled, you can then login to the appliance CLI, which is where the real troubleshooting can begin.

Troubleshooting VMware HCX

You’ll want to login to the appliance using “admin” as the user name and the password entered when you installed the appliance.  SU to “root” and enter the “root” password.  This gives you access to the appliance, which has a limited set of Linux commands.

You’ll want to enter the HCX Central CLI (CCLI) to use the HCX commands.  Since you’re already logged in as “root,” you just type “ccli” at the command prompt.  After you’re in the CCLI, you can type “help” to get a list of commands.

One of the first tests to run would be the Health Checker. Type “hc” at the command prompt, and the HCX manager will run through a series of tests to check on the health of the environment.

“list” will give you a list of the HCX appliances that have been deployed.

You’ll want to connect to an appliance to run the commands specific to that appliance.  As shown above, if you want to connect to the Interconnect appliance, you would type “go 0,” which would connect you to node 0.  From here, you can run a ton of commands, such as “show ipsec status,” which will show a plethora of information related to the tunnel.  Type “q” to exit this command.

You can also run the Health Check on this node from here, export a support bundle (under the debug command), and a multitude of other “show” commands.  Under the “show” command, you can get firewall information, flow runtime information, and a lot of other useful troubleshooting info.

If you need to actually get on the node and run Linux commands for troubleshooting, you’ll enter “debug remoteaccess enable,” which enables SSH on the remote node.  Then you can just type “ssh” and it will connect you to the interconnect node.

Have questions about this process? Contact us or leave a reply.

-Michael Moore, Associate Cloud Consultant

Facebooktwitterlinkedinmailrss

Automating Windows Server Builds with Packer

It might not be found on a curated list of urban legends, but trust me, IT IS possible (!!!) to fully automate the building and configuration of Windows virtual machines for AWS.  While Windows Server may not be your first choice of operating systems for immutable infrastructure, sometimes you as the IT professional are not given the choice due to legacy environments, limited resources for re-platforming, corporate edicts, or lack of internal *nix knowledge.

Complain all you like, but after the arguing, fretting, crying, and gnashing of teeth is done, at some point you will still need to build an automated deployment pipeline that requires Windows Server 2016.  With some PowerShell scripting and HashiCorp Packer, it is relatively easy and painless to securely build and configure Windows AMIs for your particular environment regardless of environment variables.

Configuration Options

Let’s dig into an example of how to build a custom configured Windows Server 2016 AMI.  You will need access to an AWS account and have sufficient permissions to create and manage EC2 instances.  If you need an AWS account, you can create one for free.

I am using VS Code and built-in terminal with Windows Subsystem for Linux to create the Packer template and run Packer, however Packer is available for several many Linux distros, Mac OS, and Windows.

First, download and unzip Packer:

/*
~/packer-demo:\> wget https://releases.hashicorp.com/packer/1.3.4/packer_1.3.4_linux_amd64.zip 
https://releases.hashicorp.com/packer/1.3.4/packer_1.3.4_linux_amd64.zip 
Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.1.183, 151.101.65.183, 151.101.129.183, ... 
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.1.183|:443... connected. 
HTTP request sent, awaiting response... 200 OK 
Length: 28851840 (28M) [application/zip] 
Saving to: ‘packer_1.3.4_linux_amd64.zip’ 
‘packer_1.3.4_linux_amd64.zip’ saved [28851840/28851840] 
~/packer-demo:\> unzip packer_1.3.4_linux_amd64.zip 
Archive:  packer_1.3.4_linux_amd64.zip   
 inflating: packer
*/

Now that we have Packer unzipped, verify that it is executable by checking the version:

/*
~/packer-demo:\> packer --version
1.3.3
*/

Packer can build machine images for a number of different platforms.  We will focus on the amazon-ebs builder, which will create an EBS-backed AMI.  At a high level, Packer performs these steps:

  1. Read configuration settings from a json file
  2. Uses the AWS API to stand up an EC2 instance
  3. Connect to the instance and provision it
  4. Shut down and snapshot the instance
  5. Create an Amazon Machine Image (AMI) from the snapshot
  6. Clean up the mess

The amazon-ebs builder can create temporary keypairs, security group rules, and establish a basic communicator for provisioning.  However, in the interest of tighter security and control, we will want to be prescriptive for some of these settings and use a secure communicator to reduce the risk of eavesdropping while we provision the machine.

There are two communicators Packer uses to upload scripts and files to a virtual machine: SSH (the default) and WinRM.  While there is a nice Win32 port of OpenSSH for Windows, it is not currently installed by default on Windows machines, but WinRM is available natively in all current versions of Windows, so we will use that to provision our Windows Server 2016 machine.

Let’s create and edit the userdata file that Packer will use to bootstrap the EC2 instance:

/*
 
# USERDATA SCRIPT FOR AMAZON SOURCE WINDOWS SERVER AMIS
# BOOTSTRAPS WINRM VIA SSL
 
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope LocalMachine -Force -ErrorAction Ignore
$ErrorActionPreference = "stop"
 
# Remove any existing Windows Management listeners
Remove-Item -Path WSMan:\Localhost\listener\listener* -Recurse
 
# Create self-signed cert for encrypted WinRM on port 5986
$Cert = New-SelfSignedCertificate -CertstoreLocation Cert:\LocalMachine\My -DnsName "packer-ami-builder"
New-Item -Path WSMan:\LocalHost\Listener -Transport HTTPS -Address * -CertificateThumbPrint $Cert.Thumbprint -Force
 
# Configure WinRM
cmd.exe /c winrm quickconfig -q
cmd.exe /c winrm set "winrm/config" '@{MaxTimeoutms="1800000"}'
cmd.exe /c winrm set "winrm/config/winrs" '@{MaxMemoryPerShellMB="1024"}'
cmd.exe /c winrm set "winrm/config/service" '@{AllowUnencrypted="false"}'
cmd.exe /c winrm set "winrm/config/client" '@{AllowUnencrypted="false"}'
cmd.exe /c winrm set "winrm/config/service/auth" '@{Basic="true"}'
cmd.exe /c winrm set "winrm/config/client/auth" '@{Basic="true"}'
cmd.exe /c winrm set "winrm/config/service/auth" '@{CredSSP="true"}'
cmd.exe /c winrm set "winrm/config/listener?Address=*+Transport=HTTPS" "@{Port=`"5986`";Hostname=`"packer-ami-builder`";CertificateThumbprint=`"$($Cert.Thumbprint)`"}"
cmd.exe /c netsh advfirewall firewall add rule name="WinRM-SSL (5986)" dir=in action=allow protocol=TCP localport=5986
cmd.exe /c net stop winrm
cmd.exe /c sc config winrm start= auto
cmd.exe /c net start winrm
 
*/

There are four main things going on here:

  1. Set the execution policy and error handling for the script (if an error is encountered, the script terminates immediately)
  2. Clear out any existing WS Management listeners, just in case there are any preconfigured insecure listeners
  3. Create a self-signed certificate for encrypting the WinRM communication channel, and then bind it to a WS Management listener
  4. Configure WinRM to:
    • Require an encrypted (SSL) channel
    • Enable basic authentication (usually this not secure as the password goes across in plain text, but we are forcing encryption)
    • Configure the listener to use port 5986 with the self-signed certificate we created earlier
    • Add a firewall rule to open port 5986

Now that we have userdata to bootstrap WinRM, let’s create a Packer template:

/*
~/packer-demo:\> touch windows2016.json
*/

Open this file with your favorite text editor and add this text:

/*
{
"variables": {
"build_version": "{{isotime \"2006.01.02.150405\"}}",
"aws_profile": null,
"vpc_id": null,
"subnet_id": null,
"security_group_id": null
},
"builders": [
{
"type": "amazon-ebs",
"region": "us-west-2",
"profile": "{{user `aws_profile`}}",
"vpc_id": "{{user `vpc_id`}}",
"subnet_id": "{{user `subnet_id`}}",
"security_group_id": "{{user `security_group_id`}}",
"source_ami_filter": {
"filters": {
"name": "Windows_Server-2016-English-Full-Base-*",
"root-device-type": "ebs",
"virtualization-type": "hvm"
},
"most_recent": true,
"owners": [
"801119661308"
]
},
"ami_name": "WIN2016-CUSTOM-{{user `build_version`}}",
"instance_type": "t3.xlarge",
"user_data_file": "userdata.ps1",
"associate_public_ip_address": true,
"communicator": "winrm",
"winrm_username": "Administrator",
"winrm_port": 5986,
"winrm_timeout": "15m",
"winrm_use_ssl": true,
"winrm_insecure": true
}
]
}
*/

Couple of things to call out here.  First, the variables block at the top references some values that are needed in the template.  Insert values here that are specific to your AWS account and VPC:

  • aws_profile: I use a local credentials file to store IAM user credentials (the file is shared between WSL and Windows). Specify the name of a credential block that Packer can use to connect to your account.  The IAM user will need permissions to create and modify EC2 instances, at a minimum
  • vpc_id: Packer will stand up the instance in this VPC
  • aws_region: Your VPC should be in this region. As an exercise, change this value to be set by a variable instead
  • user_data_file: We created this file earlier, remember? If you saved it in another location, make sure the path is correct
  • subnet_id: This should belong to the VPC specified above. Use a public subnet if needed if you do not have a Direct Connect
  • security_group_id: This security group should belong to the VPC specified above. This security group will be attached to the instance that Packer stands up.  It will need, at a minimum, inbound TCP 5986 from where Packer is running

Next, let’s validate the template to make sure the syntax is correct, and we have all required fields:

/*
~/packer-demo:\> packer validate windows2016.json
Template validation failed. Errors are shown below.
Errors validating build 'amazon-ebs'. 1 error(s) occurred:
* An instance_type must be specified

Whoops, we missed instance_type.  Add that to the template and specify a valid EC2 instance type.  I like using beefier instance types so that the builds get done quicker, but that’s just me.

            "ami_name": "WIN2016-CUSTOM-{{user `build_version`}}",
"instance_type": "t3.xlarge",
"user_data_file": "userdata.ps1",
*/

Now validate again:

/*
~/packer-demo:\> packer validate windows2016.json
Template validated successfully.
*/

Awesome.  Couple of things to call out in the template:

  • source_ami_filter: we are using a base Amazon AMI for Windows Server 2016 server. Note the wildcard in the AMI name (Windows_Server-2016-English-Full-Base-*) and the owner (801119661308).  This filter will always pull the most recent match for the AMI name.  Amazon updates its AMIs about once a month
  • associate_public_ip_address: I specify true because I don’t have Direct Connect or VPN from my workstation to my sandbox VPC. If you are building your AMI in a public subnet and want to connect to it over the internet, do the same
  • ami_name: this is a required property, and it must also be unique within your account. We use a special function here (isotime) to generate a timestamp, ensuring that the AMI name will always be unique (e.g. WIN2016-CUSTOM-2019.03.01.000042)
  • winrm_port: the default unencrypted port for WinRM is 5985. We disabled 5985 in userdata and enabled 5986, so be sure to call it out here
  • winrm_use_ssl: as noted above, we are encrypting communication, so set this to true
  • winrm_insecure: this is a rather misleading property name. It really means that Packer should not check if encryption certificate is trusted.  We are using a self-signed certificate in userdata, so set this to true to skip certificate validation

Now that we have our template, let’s inspect it to see what will happen:

/*
~/packer-demo:\> packer inspect windows2016.json
Optional variables and their defaults:
aws_profile       = default
build_version     = {{isotime "2006.01.02.150405"}}
security_group_id = sg-0e1ca9ba69b39926
subnet_id         = subnet-00ef2a1df99f20c23
vpc_id            = vpc-00ede10ag029c31e0
Builders:
amazon-ebs
Provisioners:
 
Note: If your build names contain user variables or template
functions such as 'timestamp', these are processed at build time,
and therefore only show in their raw form here.
*/

Looks good, but we are missing a provisioner!  If we ran this template as is, all that would happen is Packer would make a copy of the base Windows Server 2016 AMI and make you the owner of the new private image.

There are several different Packer provisioners, ranging from very simple (the windows-restart provisioner just reboots the machine) to complex (the chef client provisioner installs chef client and does whatever Chef does from there).  We’ll run a basic powershell provisioner to install IIS on the instance, reboot the instance using windows-restart, and then we’ll finish up the provisioning by executing sysprep on the instance to generalize it for re-use.

We will use the PowerShell cmdlet Enable-WindowsOptionalFeature to install the web server role and IIS with defaults, add this text to your template below the builders [ ] section:

/*
"provisioners": [
{
"type": "powershell",
"inline": [
"Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebServerRole",
"Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebServer"
]
},
{
"type": "windows-restart",
"restart_check_command": "powershell -command \"& {Write-Output 'Machine restarted.'}\""
},
{
"type": "powershell",
"inline": [
"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\InitializeInstance.ps1 -Schedule",
"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\SysprepInstance.ps1 -NoShutdown"
]
}
]
*/

Couple things to call out about this section:

  • The first powershell provisioner uses the inline Packer simply appends these lines in order into a file, then transfers and executes the file on the instance using PowerShell
  • The second provisioner, windows-restart, simply reboots the machine while Packer waits. While this isn’t always necessary, it is helpful to catch instances where settings do not persist after a reboot, which was probably not your intention
  • The final powershell provisioner executes two PowerShell scripts that are present on Amazon Windows Server 2016 AMIs and part of the EC2Launch application (earlier versions of Windows use a different application called EC2Config). They are helper scripts that you can use to prepare the machine for generalization, and then execute sysprep

After validating your template again, let’s build the AMI!

/*
~/packer-demo:\> packer validate windows2016.json
Template validated successfully.
*/
/*
~/packer-demo:\> packer build windows2016.json
amazon-ebs output will be in this color.
==> amazon-ebs: Prevalidating AMI Name: WIN2016-CUSTOM-2019.03.01.000042
amazon-ebs: Found Image ID: ami-0af80d239cc063c12
==> amazon-ebs: Creating temporary keypair: packer_5c78762a-751e-2cd7-b5ce-9eabb577e4cc
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Instance ID: i-0384e1edca5dc90e5
==> amazon-ebs: Waiting for instance (i-0384e1edca5dc90e5) to become ready...
==> amazon-ebs: Waiting for auto-generated password for instance...
amazon-ebs: It is normal for this process to take up to 15 minutes,
amazon-ebs: but it usually takes around 5. Please wait.
amazon-ebs:  
amazon-ebs: Password retrieved!
==> amazon-ebs: Using winrm communicator to connect: 34.220.235.82
==> amazon-ebs: Waiting for WinRM to become available...
amazon-ebs: WinRM connected.
amazon-ebs: #> CLIXML
amazon-ebs: System.Management.Automation.PSCustomObjectSystem.Object1Preparing modules for first 
use.0-1-1Completed-1 1Preparing modules for first use.0-1-1Completed-1 
==> amazon-ebs: Connected to WinRM!
==> amazon-ebs: Provisioning with Powershell...
==> amazon-ebs: Provisioning with powershell script: /tmp/packer-powershell-provisioner561623209
amazon-ebs:
amazon-ebs:
amazon-ebs: Path          :
amazon-ebs: Online        : True
amazon-ebs: RestartNeeded : False
amazon-ebs:
amazon-ebs: Path          :
amazon-ebs: Online        : True
amazon-ebs: RestartNeeded : False
amazon-ebs:
==> amazon-ebs: Restarting Machine
==> amazon-ebs: Waiting for machine to restart...
amazon-ebs: Machine restarted.
amazon-ebs: EC2AMAZ-LJV703F restarted.
amazon-ebs: #> CLIXML
amazon-ebs: System.Management.Automation.PSCustomObjectSystem.Object1Preparing modules for first 
use.0-1-1Completed-1 
==> amazon-ebs: Machine successfully restarted, moving on
==> amazon-ebs: Provisioning with Powershell...
==> amazon-ebs: Provisioning with powershell script: /tmp/packer-powershell-provisioner928106343
amazon-ebs:
amazon-ebs: TaskPath                                       TaskName                          State
amazon-ebs: --------                                       --------                          -----
amazon-ebs: \                                              Amazon Ec2 Launch - Instance I... Ready
==> amazon-ebs: Stopping the source instance...
amazon-ebs: Stopping instance, attempt 1
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating unencrypted AMI WIN2016-CUSTOM-2019.03.01.000042 from instance i-0384e1edca5dc90e5
amazon-ebs: AMI: ami-0d6026ecb955cc1d6
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.
 
==> Builds finished. The artifacts of successful builds are:
==> amazon-ebs: AMIs were created:
us-west-2: ami-0d6026ecb955cc1d6
*/

The entire process took approximately 5 minutes from start to finish.  If you inspect the output, you can see where the first PowerShell provisioner installed IIS (Provisioning with powershell script: /tmp/packer-powershell-provisioner561623209), where sysprep was executed (Provisioning with powershell script: /tmp/packer-powershell-provisioner928106343) and where Packer yielded the AMI ID for the image (ami-0d6026ecb955cc1d6) and cleaned up any stray artifacts.

Note: most of the issues that occur during a Packer build are related to firewalls and security groups.  Verify that the machine that is running Packer can reach the VPC and EC2 instance using port 5986.  You can also use packer build -debug win2016.json to step through the build process manually, and you can even connect to the machine via RDP to troubleshoot a provisioner.

Now, if you launch a new EC2 instance using this AMI, it will already have IIS installed and configured with defaults.  Building and securely configuring Windows AMIs with Packer is easy!

For help getting started securely building and configuring Windows AMIs for your particular environment, contact us.

-Jonathan Eropkin, Cloud Consultant

Facebooktwitterlinkedinmailrss