At Sumo Logic, we use Amazon Web Services (AWS) for everything. Our product, as well as all our internal infrastructure live in AWS. In this series of posts, we’ll share some useful practices around using AWS. In the first installment, I’ll outline some useful things we do with tags.
1. Organize resources
We’ve decided on a hierarchical way of managing our EC2 (Elastic Compute Cloud) resources:
Deployment
+ Cluster
+ Instance/Node
Within an AWS account, we can have multiple “deployments”. A deployment is a complete, independent copy of our product and uses the same architecture as our production service. Besides production, we use several smaller-scale deployments for development, testing and staging. Each deployment consists of a number of clusters, and each cluster of one or more instances.
Instances and their corresponding EBS (Elastic Block Store) volumes are tagged with Deployment, Cluster and Node tags. As an example, the third frontend node of our production deployment would be tagged like so:
Deployment=prod
Cluster=frontend
NodeNumber=3
There is also a direct mapping to DNS names. The DNS name for this node would be prod-frontend-3.
Combined with the filtering features in AWS Console (you can make any tag a column in the resource listings), this makes it very easy to navigate to a particular set of resources.
2. Display Instance Status
Tags can also be used as an easy way to display status information in the AWS console. Simply update a tag with the current status, whenever it changes.
The code that deploys our instances into EC2 updates a DeployStatus tag whenever it progresses from one step to another. For example, it could read:
2012-05-10 17:53 Installing Cassandra
This allows you to see what’s going on with instances at a glance.
3. Remember EBS Volume Devices
For EC2 instances that have multiple EBS volumes, when they need to be attached, our tools need to know which volume gets mapped to which device on the instance.
When we first create a volume, for example /dev/sdj, we create add a DeviceName tag to the volume with a value of /dev/sdj to track where it needs to be attached. Next time we attach the volume, we know it’s “proper place”.
4. Attribute and remind people of costs
All our developers are empowered to create their own AWS resources. This is a huge benefit for full-scale testing, performance evaluations, and many other use cases. Since AWS is not a charity, however, we need to manage costs tightly. In order to do this, we tag all AWS resources with an Owner tag (either by hand, or via our automated deployment tool).
To consume this tag, we have a cron job that runs daily and emails users who have active resources in AWS to remind them to shut down what they no longer require.
The subject line of the email reads “[AWS] Your current burn rate is $725.91/month!”. The body of the email contains a table with a more detailed cost breakdown. In addition, there is also a rollup email that goes out to the entire development team.
Summary
EC2 tags are extremely useful tools to track state, organize resources and store relationships between resources like instances and EBS volumes. There are a myriad more ways to use them. I hope these tips have been helpful.
