WHAT THE TAG?

Developing comprehensive tagging strategies for your cloud resources to achieve transparency in cost allocation.

Tagging as a tool to transform metadata into usable data is by no means a new concept. It may sound simple, but one should carefully consider what information to gather before adding metadata to your cloud resources. The journey to effective cost management may not be as straightforward as one thinks.

The reason is that the people deploying, using and managing resources have different motivations compared to those who plan budgets, do forecasts and join annual report discussions when it comes to analysing cloud costs. Hence, they tag accounts and resources differently (or do not tag them at all).

What's all the fuss over strategy?

Anyone with experience deploying cloud resources may wonder: why not just create a tagging list and automate the tags? The answer: this approach is not sustainable.

It is true that via policies, tag assignments can be automated when deploying new resources. But for existing resources, they must be assigned manually.

Furthermore, depending on the provider, there are certain resources that cannot be tagged at the resource level, but can be identified in other ways. However, a global differentiation in cost reporting only works if everything taggable is also tagged.

Note: tags do not work retroactively. This means that only when a tag is assigned to a resource it can be used as an information provider for this resource.

Example for AWS: The company board wants to know how much is spent on compute for testing across all teams, companies and/or projects in 2021. You have created an appropriate tag list and determined policies that assign each new EC2 instance to an "environment" (dev, ops, test) and a "project". The policies were implemented on December 24^th, 2021. In Q1 2022 you realize that your reporting is not meaningful. The reason: All resources that were running in the test environment before 24/12/2021 are excluded from your policies and do not appear in the reporting.

Thus, for a holistic cost transparency and analysis, rules must be developed that include existing and future resources. Considerations must be made on how to tag un-taggable resources. Furthermore, existing resources need to be edited to avoid information gaps.

Simply put, creating tagging lists or using random tags carries the risk that this effort has to be repeated.

Identify your needs by thinking about the big picture.

This is where the development of thought through tagging strategies comes into place. The purpose of a tagging strategy is to bring useful, company specific business logic into the metadata as well as bringing clear processes and standards to tagging activities across teams and projects.

Your usual cloud bill does not provide this information. This partly because cloud providers have an economic incentive to let you continue running your unused/underused resources. Therefore, you will have to carefully think about what information you need to achieve cost transparency.

You can use common cost allocation tags such as “cost center” and “project” to differentiate between projects, departments, customers, etc. for billing reports.

They can help provide information about

• which business units produce costs, to be able to allocate costs faster and better plan budgets.
• which projects use which lease types (on-demand, spots, reservation) to evaluate (and plan) whether the right type is being used.
• what projects or teams produce cloud costs over budget.
• which teams/projects are experiencing (cost) anomalies, in order to be able to understand more quickly why, for example, an unusually large amount of compute power was needed.

You can also think of using tags for automation. For example, to turn off resources in test environments (e.g. with the tag “test”) every night or on weekends in order to save costs.

For example, at Liquid Reply we created a tagging policy for resources that adds the security level of a project (dedicated as Security Zone) for one of our clients. This tagging makes it possible, for example, to track what resources at which costs are used per IT security level.

When it comes to budget, think of your ability to answer questions like “How much do our test environments cost for each project? What about companywide?”

The use cases can be very company-specific and when applied correctly, represent a real game changer in the area of cloud cost management.

Determine your tagging needs by asking the right questions.

It is important to think beyond the individual teams and IT projects and bring in the management perspective. Asking the right questions can help stakeholders to use the right tags.

Use the following questions to determine what team members you need to involve and define the information you need to create resource tags for:

• Who might need cloud cost related information in your company?
• What kind of information is relevant?
• How are budgets planned? Does the department, the team, the individual owner play a role?
• Which information has priority for cost allocation (e.g. cost center/business unit, project name or customer name)?
• Were there any problems with cost allocation in the annual financial statements in the past? If so, where were the hurdles and what info was missing?
• Are certain tasks dependent on the costs they produce? For example, is only a certain budget allocated companywide for testing?
• Do your naming- and tagging policies need to integrate with existing policies within your company?
• Does tagging need to represent details such regulatory compliance requirements for a resource? What about operational details such as uptime requirements, patching schedules, or security requirements?
• What tags will be required for all resources based on centralized IT policy? What tags will be optional? Are individual teams allowed to implement their own custom tagging schemes?

During these conversations, things like "would be nice to have" tags and information may also come up. This is usually a sign of someone recognising the power of tagging.

Best practice here is to not to be tempted to fully exploit all the possibilities of business logic. Tagging everything in fine detail will create a lot of additional work in the future.

Keep it simple and limited to the amount of information you really need.

After all, the longer the list of tags, the higher the probability that your developers, engineers and operators will run out of steam halfway through.

Consider what you got.

It often also helps to look at what tags are already being used - even if tagging is not being used consistently across all accounts and resources.

First of all, to ensure that the information that teams need is considered in the tagging strategy.

Second, it is always worth carefully considering using and reusing the terms and tags already added by developers and administrators. By using these as a basis for a new tagging plan it will help avoid adding unneeded complexity (notation, meaning) moving forward for the people seeing the tags every day.

When it comes to naming tags, two best practices have been established:

1. Use keys that are clear and self-explanatory.

1. Use the terms you use in the normal business context (example: if you never talk about business units in daily business, but about groups, teams or departments, it makes no sense to use the tag “business unit” – even if often recommended).

Create a standardised tag list.

Once you agreed on the information needed it has to be translated into tags.

A list of all keys to be used and the possible values as well as a short explanation of them helps to standardise the process and prevent different tags with the same meaning.

For cost allocation, the following tag list has proven to be best practice:

Once you finalized your tag list, engage the people who deploy resources. Obtain feedback on the things like the accuracy, spelling, ordering, and uniqueness of key and/or value designations from those who need to use them on a daily basis.

Put your tags to the test before rollout.

Start small with one Cloud provider and 3-5 obvious areas that produce costs that you want to track. Test your tag list – manually or by developing organizational policies for these areas.

Test your tag list with your determined teams for 1-2 weeks and pull a cost report by the end of your testing period. Perform a cost analysis on tag basis and discuss this information with the stakeholders that you initially developed the requirements with.

Simultaneously audit how the tagging worked on operation level with your technical teams.

Create a bug report: Where there any problems with the defined tags, the tagging process or the analysis?

Iterate, if necessary.

Final thoughts regarding the untaggables.

Before provider- or enterprise-wide rollout, it must be considered that not all cloud resources can be tagged. To avoid errors due to policies or process enforced tagging, the un-taggable resources should be identified.

In Cloud Cost Management best practices, cost allocation is applied using a combination of tags and hierarchy-based marking.

Of course, we always strive for 100% coverage - even when it comes to tagging.

In reality, the effort is not always worth it. So before considering complex structures at the hierarchy level, it is advisable to first check how many of the resources used actually belong to the un-taggable ones. The major providers such as AWS and Azure offer detailed documentation and lists of which resources are not taggable.

Then discuss based on the listing of identified untagged resources, if they need to be covered and if an account hierarchy would add value.

Resources:

Storment, J: Cloud Finops: Collaborative, Real-Time Cloud Financial Management

AWS Blog: Cost Tagging and Reporting with AWS Organizations

ServerSide: Tag Writing

Inter-University Computation Center: Tags for Cloud Resources

DevOps Blog: Best Practices for AWS Tagging with Yor

Microsoft: Resource Tagging