CMP Archives - 2nd Watch

Best Practices: Developing a Tag Strategy for AWS Billing

Tag Strategy is key to Cost Allocation for Cloud Applications.

Have you ever been out to dinner with a group of friends and at the end of the dinner the waiter comes back with one bill? Most of us have experienced this. Depending on the group of friends it’s not a big deal and everyone drops in a credit card so the bill can be split evenly. Other times, someone invites Harry and Sally, and they scrutinize the bill line by line. Inevitably they protest that they only had one glass of wine and Sally only had the salad. You recall that Sally was a little ‘handsy’ with the sampler platter, but you sit quietly. It’s in that moment you remember, that’s why the group didn’t include Harry and Sally to last year’s New Year’s dinner. No need to start the new year with an audit, am I right?

This situation can be eerily similar in many ways to cloud billing in a large enterprise. The fact that Amazon Web Services (AWS) has changed the way that an organization uses computing resources is evident. However, AWS has also delivered on the promise of truly enabling ‘chargeback’ or ‘showback’ in the enterprise so that the business units themselves are stakeholders in what was traditionally silo’d in an IT Department budget.

Now multiple stake holders from many organizations have a stake in the cost and usage of an app that resides in AWS. Luckily there are tools like 2nd Watch’s Cloud Management Platform (CMP) that can easily provide visibility to the cost of their app, or even what their entire infrastructure is costing them at the click of a button.

2nd Watch’s CMP tools are great for showing an organization’s costs and can even be used to set budget notifications so that the business unit doesn’t inadvertently spend more than is budgeted on an environment. CMP is a powerful tool that can deliver powerful insights to your business and can be made more powerful by implementing a thorough tagging strategy.

Tag your it…

We live in a world of tags and hashtags. Seemingly overnight tags have made their way into everyday language. This is not by accident as cloud interactions with Facebook and Twitter have become so commonplace, they have altered the world’s language.

Beyond their emergence in our everyday vernacular, they have a key function. In AWS, applying tags to various cloud resources like EC2 and RDS is key to having quality accounting for allocating charges. Our team of experts at 2nd Watch can work with you to ensure that your tagging strategy is implemented in the most effective manner for your organization. After all, a tagging strategy can and will vary by organization. It depends on you and how you want to be able to report on your resources. Do you want to be able to report on your resources used by cost center, application, environment type (like dev or prod), owner, department, geographic area, or if this resource was managed by a managed service provider like 2nd Watch?

Without having a well thought out tagging strategy your invoicing discussions will sound much like the fictional dinner described above. Who pays for what and why?

Tag Strategy and Hygiene…

Implementing a sound tagging strategy at the outset when a resource or environment is deployed is the first step. At the inception it’s important to know some “gotchas” that can derail a tagging implementation. One of these is that tags are case sensitive. For example, mktg will report separately from Mktg. Also keep in mind, that in today’s ever changing business environment organizations are forced to adjust and reorganize themselves to stay competitive.

Revisiting your tagged resource strategy will need to be done from time to time to ensure tag relevance. If a stake holder moves out of a role, gets promoted, or retires from the organization altogether, you will need to stay on top of the tagging for their environment to be sure that it is still relevant to the new organization.

What about the un-taggables?

Having standardization and a tag plan works great for AWS resources like EC2 and RDS as explained before. What about untaggable resources, Network transfer charges, and items like a NAT gateway or a VPC Endpoint? There will be shared resources like these in your applications environment. It is best to review these shared untagged resources early on, and decide where to best allocate that cost.

At 2nd Watch, we have these very discussions with our clients on a regular basis. We can easily guide them through the resources associated with the app and where to allocate each cost. With a tool like CMP we can configure a client’s cost allocation hierarchy so they can view their ongoing costs in real time.

For it’s part, Amazon does a great job providing an up-to-date user guide for what resources can be tagged. Click here for great reference documentation to help while you develop your tag strategy.

Rinse and repeat as necessary

Your tagging strategy can’t be a ‘fire and forget’ pronouncement. To be effective your organization will need to enforce it on a consistent basis. For instance, as new devops personnel are brought into an organization, it will be key to insuring it stays under control.

These are the types of discussions that 2nd Watch adds a lot of value to. Our cloud expertise in AWS for large enterprises will insure that you are able to precisely account for your cloud infrastructure spend at the click of a button through CMP.

After all, we all want to enjoy our meal and move on with the next activity. Stop by and visit us at re:Invent booth #1104 for more help.

— Paul Wells, Account Manager, 2nd Watch

Budgets: The Simple Way to Encourage Cloud Cost Accountability

Controlling costs is one of the grea challenges facing IT and Finance managers today. The cloud, by nature, makes it easy to spin up new environments and resources that can cost thousands of dollars each month. And, while there are many ways to help control costs, one of the simplest and most effective methods is to set and manage cloud spend-to-budget. While most enterprise budgets are set at a business unit or department, for cloud spend, mapping that budget down to the workload can establish strong accountability within the organization.

One popular method that workload owners use to manage spend is to track month-over-month cost variances. However, if costs do not drastically increase from one month to another, this method does very little to control spend. It is only until a department is faced with budget issues that workload owners work diligently to reduce costs. That’s because, when budgets are set for each workload, owners become more aware of how their cloud spend impacts the company financials and tend to more carefully manage their costs.

In this post, we provide four easy steps to help you manage workload spend-to-budget effectively.

Step 1: Group Your Cloud Resources by Workload and Environment

Use a financial management tool such as 2nd Watch CMP Finance Manager to group your cloud resources by workload and its environment (Test, Dev, Prod). This can easily be accomplished by creating a standard where each workload/environment has its own cloud account, or by using tags to identify the resources associated with each workload. If using tags, use a tag for the workload name such as workload_name: and a tag for the environment such as environment:. More tagging best practices can be found here.

Step 2: Group Your Workloads and Environments by Business Group

Once your resources are grouped by workload/environment, CMP Finance Manager will allow you to organize your workload/environments into business groups. For example:

a. Business Group 1

i. Workload A

1. Workload A Dev

2. Workload A Test

3. Workload A Prod

ii. Workload B

1. Workload B Dev

2. Workload B Test

3. Workload B Prod

b. Business Group 2

i. Workload C

1. Workload C Dev

2. Workload C Test

3. Workload C Prod

ii. Workload D

1. Workload D Dev

2. Workload D Test

3. Workload D Prod

Step 3: Set Budgets

At this point, you are ready to set up budgets for each of your workloads (each workload/environment and the total workload as you may have different owners). We suggest you set annual budgets aligned to your fiscal year and have the tool you use programmatically recalculate the budget at the end of each month with the amount remaining in your annual budget.

Step 4: Create Alerts

The final step is to create alerts to notify owners and yourself when workloads either have exceeded or are on track to exceed the current month or annual budget amount. Here are some budget notifications we recommend:

ME forecast exceeds month budget
MTD spend exceeds MTD budget
MTD spend exceeds month budget
Daily spend exceed daily budget
YE forecast exceeds year budget
YTD spend exceeds YE budget

Once alerts are set, owners can make timely decisions regarding spend. The owner can now proactively shift to spot instances, purchase reserved instances, change instance sizes, park the environment when not in use, or even refactor the application to take advantage of cloud native services like AWS Lambda.

Our experience has shown that enterprises that diligently set up and manage spend-to-budget by workload have more control of their costs and ultimately, spend less on their cloud environments without sacrificing user experience.

–Timothy Hill, Senior Product Manager, 2nd Watch

Cost Accounting for Amazon WorkSpaces

Who would have thought, back in 2014, when AWS launched Amazon WorkSpaces it would have such an impact on the virtual desktop market? Amazon WorkSpaces—AWS’ fully managed, secure desktop computing service—allows enterprises to easily provision cloud-based virtual desktops and provide users access to the documents, applications, and resources they need from any supported device. Over these three short years, Amazon WorkSpaces has made great strides in reducing the costs related to VDI deployment, support and software packaging while improving service levels and deployment time of new applications. Amazon WorkSpaces provides the flexibility to securely work from anywhere, anytime and on any device without the cost and complexity of traditional VDI infrastructure.

However, enterprises have faced a few challenges when deploying Amazon WorkSpaces. One of the grea challenges with wholesale deployment of Amazon WorkSpaces has been how to allocate the costs associated with thousands of instances to the various departments that are using each resource. In 2016 AWS enabled users to tag each workspace with up to 50 tags. While this is a step in the right direction, tagging is not included in the launch process. Instead, users have to remember to tag the instance after it is launched. This is where the process tends to break down, leaving thousands of dollars related to cloud spend either un-allocated or incorrectly allocated.

To address this drawback, it is important to create and implement two processes. The first step is pretty basic: Develop a process and train all team members responsible for launching new WorkSpaces to tag each workspace after it is launched. The second step is to set up automation to efficiently audit and provide notifications when resources (specifically Amazon WorkSpaces) are launched without a particular tag or set of tags. Unfortunately, with Amazon WorkSpaces you aren’t able to use the AWS Config “required-tags” rule to enforce your process policy as Config only supports a limited set of AWS resource types. (NOTE: You can check out the AWS Config Developer Guide for more on using it to enforce tag requirements on Config supported resources.) Instead, you can roll your own tag enforcement solution using AWS Lambda and CloudTrail.

This process is fairly simple. When you activate AWS CloudTrail logs, AWS will dump all API calls as JSON log files to an S3 bucket. You can then setup a trigger on that bucket to invoke an AWS Lambda function that can scan the logs for specific events, such as Amazon WorkSpace’s “CreateWorkSpaces” method. If it finds an event, it can publish a message to an SNS topic notifying you that the resource does not have the appropriate tag. You can even set the message up to include the creator tag that AWS adds to all new resources. This way, if you need to know who launched the instance in order to determine how to tag it, you will have that information included.

Even when you have the tag in place there is still the issue of how to allocate those costs incurred before the resource was tagged. Because AWS tags are point in time, only costs associated after the tag is in place will be included in any AWS tag report. 2nd Watch’s cloud financial management tool, CMP|FM, is a powerful resource that can provide accurate cost accounting and deep, financial insight into Amazon WorkSpaces usage by applying boundaries by month to all tags. In other words, any tag applied during the middle of the month will be applied to the entire month’s usage— appropriately accounting for all of your costs associated with Amazon WorkSpaces—without the need to manually allocate them to the correct department.

If you are looking to deploy Amazon WorkSpaces across your enterprise, it is important to ensure that you have the systems in place for proper cost accounting. This includes implementing documented processes for tagging during launch and automation to identify and manage untagged instances, and leveraging powerful tools like 2nd Watch CMP|FM for all your cost allocation needs to ensure accurate cost accounting.

— Timothy Hill, Senior Product Manager, 2nd Watch