We started using HBase on EC2 sometime back in 2009. We thought that our data is important and we should have an option of restoring the data. We attached EBS volumes to our HBase nodes and configured HBase and Hadoop installation to store all the data on the attached EBS volumes. Then came the concept [...]
A sunny Sunday morning. I am preparing to go out with my wife. And suddenly a pingdom alert comes to my phone. The website is down! I run to my ubuntu desktop and hurriedly open ylastic, amazon aws console, splunk etc. Console shows nothing unusual. The instance on which the website is running is shown [...]
I think most of the EC2 users know about EC2′s instance metadata query tool. It’s an executable you can install on your ec2 instance. Once you save the file and make it executable you can get metadata of the instance such as hostname, block-device-mapping, instance-id etc. But there is a web service version of this [...]
At GumGum we are using autoscaling successfully. Choosing the right metrics for autoscaling is an ongoing process as your cluster and applications change. When we researched which metrics to use for autoscaling we found very little literature in the blogosphere. That is why I decided to document our experiences with it.
We are running a HBase (Currently 0.20.4) cluster on ec2. I thought it will be useful for others to know some tips about running HBase in EC2. 1) Use private dns addresses in config files such as hdfs-site.xml, hbase-site.xml. On ec2 Ubuntu instances java’s getHost() gets resolved to the private dns addresses. 2) Use c1.xlarge [...]
Amazon recently announced the AWS SDK for java. SDK or a java api is very much needed – especially if you are writing your automation scripts in groovy. We have tried multiple java apis in our scripts including JetS3t and Typica. These apis were really helpful, but they only supported some of the AWS services [...]
