To begin with let’s look at an autoscaling trigger creation command:

as-create-or-update-trigger my-cpu --auto-scaling-group myautoscalinggroup --dimensions "AutoScalingGroupName=myautoscalinggroup" --measure CPUUtilization --period 60 --statistic Average --lower-threshold 20 --upper-threshold 50 --breach-duration 300 --upper-breach-increment 1 --lower-breach-increment 1

The dimensions parameter indicates that this is a trigger created on myautoscalinggroup autoscaling group. The measure indicates that it’s a trigger based on CPU Utilization. Period indicates that the trigger will be evaluated every 60 seconds. if upper threshold (average cpu utilization goes above 50%) is breached for 300 seconds then start 1 additional instance. If lower threshold (average cpu utilization goes below 20%) is breached for 300 seconds then 1 instance is scaled down.

The metrics on which a autoscaling trigger is built (Cloudwatch measure) is the most important decision you make when you set up an autoscaling cluster.

There are many metrics you can choose to scale up or down your cluster. CPU utilization, latency, network out bytes, disk reads in bytes etc. You can set up autoscaling based on whatever data Cloudwatch offers.

Latency or CPU utilization were obvious choices for our web cluster. We started out with latency as the metrics for autoscaling. Quickly we realized that latency was not the right metrics for us. Our application was making calls to some third party web services. Sometimes third party webservices problems caused our latency to go up. In that case, adding more instances to the cluster didn’t help much. Furthermore, third party calls made our latency spike unevenly. We wanted our cluster to go up smoothly as the traffic increases and come down smoothly as the traffic decreased.

We decided to switch to CPU Utilization as the metric for autoscaling. We started thinking about our upper and lower thresholds. We chose 50 as the upper threshold. It can take autoscaling few minutes to kick in. It was important for us to keep enough room (50%) for CPU to go up in those few minutes. In fact by setting a breach duration of 300 seconds we ensured that autoscaling did not kick in immediately. This adjustment accommodated occasional traffic spikes that went down on their own. By looking at our average CPU utilization, we decided to keep the lower threshold at 20% CPU utilization. This worked well when we had a relatively small cluster. Our traffic pattern caused dramatic increase in our average CPU utilization in a smaller cluster (3 to 5 instances). CPU went up to 50% quickly and came back down to 20% when traffic subsided.

Recently our traffic increased dramatically and as a result our cluster increased tenfold! Now we were running 40 instances! Our CTO observed that the autoscaling was adding instances when they are necessary, but the cluster was not scaling down quickly enough. In a bigger cluster, average cpu was fairly stable and was not undergoing dramatic changes as in a smaller cluster. So our CTO decided to adjust the lower threshold to 40% of CPU Utilization. This worked and the autoscaling cluster started scaling down.

You can see the trigger scaling instances up and down throughout a day below. Time is on x axis and number of instances is on y axis.

We hope our experiences are helpful to others implementing autoscaling. I am sure we will have to revisit these numbers when our cluster grows to 100s of instances! I will write a new blog post describing my experiences then.

SDK or a java api is very much needed – especially if you are writing your automation scripts in groovy. We have tried multiple java apis in our scripts including JetS3t and Typica. These apis were really helpful, but they only supported some of the AWS services and were not up to date (for obvious reasons). Having one java api that can support all of AWS technologies was definitely the need of the hour. I am sure Amazon will keep it updated as new services are released. They have the necessary resources to do so.

Furthermore, Amazon has also uploaded the SDK to the maven repository.
You can use the following dependency in your pom.xml:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.0.002</version>
</dependency>

The java doc for the SDK is hosted at http://docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/index.html

Amazon has also opened the SDK source code for all. They have mirrored the SDK code repository at github. You can look at the SDK code at http://github.com/amazonwebservices/aws-sdk-for-java

Keep machines in the same availability zone
Don’t scatter your machines that talk to each other across multiple availability zones. You will end up paying for the bandwidth. Off course this does not apply to the people who purposely keep their machines in different availibilty zones to ensure high availibility.

Use cnames instead of A records
In other words do not map your domain name to an elastic ip. Map it to the public domain name of the instance. Let’s say you have a machine with name splunk.gumgum.com. All the web servers (within the same availability zone) send considerable data to this machine. If you setup splunk.gumgum.com as an A record, your data will go out and come back in. But if you map it as cname, your data will always remain within the ec2 cluster. To read more about this visit Eric Hammond’s Using Elastic IP to Identify Internal Instances on Amazon EC2 post.

Use spot instances
Use spot instances whenever possible. Spot instances cost less than on demand instances. You can read more about spot instances on Amazon’s web page. There is a website that shows spot history price graphs. It will tell you an average price of various types of instances. The average price of a m1.small sport instance is 3 cents an hour. You pay 8.5 cents an hour for the same type of on demand instance.

Choose your instance types wisely
A smallest possible instance may not be the right choice to save money. For example c1.medium costs twice as much as m1.small. But it offers 5 times more compute power than m1.small. Thus in some compute heavy jobs it might be cheaper to use c1.medium instead of the least expensive m1.small.

Choose the smallest possible storage
Choose the smallest possible EBS volumes and RDS instances. It is very easy to expand an ebs volume when needed. Same is the case with a RDS instance. Furthermore by choosing smaller drives, you will save money needed for the backup(snapshot) storage too.

Use Autoscaling
Autoscaling can start additional instances when needed. It can shut down the additional instances when the need disappears. This is especailly useful for hosting websites/web services that have a specific traffic/usage pattern.

Reserve your instances
You can reduce your costs significantly by reserving number of instances for a year or for three years. Reservation is only a billing concept and thus you don’t have to designate any istances as reserved. If you reserve 4 m1.small instances, Amazon will make 4 of your running m1.small instances as reserved. For more information visit Werner Vogels’s (Amazon CTO) blog.

There might be numerous other ways you can save money. Please feel free to leave a comment to share your ways. The more the merrier!

Auto Scaling allows you to automatically scale your Amazon EC2 capacity up or down according to conditions you define.

Simply put, it can start and stop ec2 instances based on a condition you define. This is especially important in an environment where you pay more for more capacity. Thus if a website has a defined traffic pattern such as traffic peaks every afternoon and goes down in the night, you can have autoscaling start more web servers in the afternoon and shut down unnecessary ones in the night. Isn’t that amazing?

You need to be using an elastic load balancer to configure auto scaling. Configuring autoscaling consists of three steps:
1) Create launch configuration
2) Create auto scaling group
3) Create auto scaling trigger

Create launch configuration
Launch configuration provides information about the instance to be launched. You can see that an AMI, instance type, availability zone and security groups are specified when a luanch configuration is created.

as-create-launch-config web --image-id ami-656583ff --instance-type m1.small --group ssh,web --key my-keypair


Create autoscaling group
A launch configuration is attached with an autoscaling group. The group consists of pool of instances. You can specify minimum and maximum size of the pool

as-create-auto-scaling-group myasgroup --launch-configuration web --availability-zones us-east-1a --min-size 4 --max-size 10 --cooldown 120 --load-balancers myloadbalancer


Create autoscaling trigger
The trigger defines the conditions to start or stop ec2 instances. For example:

as-create-or-update-trigger my-latency-trigger --auto-scaling-group myasgroup --dimensions "LoadBalancerName=myloadbalancer" --measure Latency --period 60 --statistic Average --lower-threshold 0.25 --upper-threshold 0.75 --breach-duration 300 --lower-breach-increment=-1 --upper-breach-increment 1 --namespace "AWS/ELB" --unit Seconds

will create a latency trigger with 0.25 as the lower threshold and 0.75 seconds as the upper threshold. If your website latency goes beyond 0.75 seconds, this trigger will be activated and hence an instance will be launched (as specified by upper breach increment). Besides latency you can use different dimensions monitored by cloudwatch such as request count, health host count and unhealthy host count.

That’s it! It’s that simple. More information about autoscaling can be read here.