SDK or a java api is very much needed – especially if you are writing your automation scripts in groovy. We have tried multiple java apis in our scripts including JetS3t and Typica. These apis were really helpful, but they only supported some of the AWS services and were not up to date (for obvious reasons). Having one java api that can support all of AWS technologies was definitely the need of the hour. I am sure Amazon will keep it updated as new services are released. They have the necessary resources to do so.

Furthermore, Amazon has also uploaded the SDK to the maven repository.
You can use the following dependency in your pom.xml:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.0.002</version>
</dependency>

The java doc for the SDK is hosted at http://docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/index.html

Amazon has also opened the SDK source code for all. They have mirrored the SDK code repository at github. You can look at the SDK code at http://github.com/amazonwebservices/aws-sdk-for-java

Keep machines in the same availability zone
Don’t scatter your machines that talk to each other across multiple availability zones. You will end up paying for the bandwidth. Off course this does not apply to the people who purposely keep their machines in different availibilty zones to ensure high availibility.

Use cnames instead of A records
In other words do not map your domain name to an elastic ip. Map it to the public domain name of the instance. Let’s say you have a machine with name splunk.gumgum.com. All the web servers (within the same availability zone) send considerable data to this machine. If you setup splunk.gumgum.com as an A record, your data will go out and come back in. But if you map it as cname, your data will always remain within the ec2 cluster. To read more about this visit Eric Hammond’s Using Elastic IP to Identify Internal Instances on Amazon EC2 post.

Use spot instances
Use spot instances whenever possible. Spot instances cost less than on demand instances. You can read more about spot instances on Amazon’s web page. There is a website that shows spot history price graphs. It will tell you an average price of various types of instances. The average price of a m1.small sport instance is 3 cents an hour. You pay 8.5 cents an hour for the same type of on demand instance.

Choose your instance types wisely
A smallest possible instance may not be the right choice to save money. For example c1.medium costs twice as much as m1.small. But it offers 5 times more compute power than m1.small. Thus in some compute heavy jobs it might be cheaper to use c1.medium instead of the least expensive m1.small.

Choose the smallest possible storage
Choose the smallest possible EBS volumes and RDS instances. It is very easy to expand an ebs volume when needed. Same is the case with a RDS instance. Furthermore by choosing smaller drives, you will save money needed for the backup(snapshot) storage too.

Use Autoscaling
Autoscaling can start additional instances when needed. It can shut down the additional instances when the need disappears. This is especailly useful for hosting websites/web services that have a specific traffic/usage pattern.

Reserve your instances
You can reduce your costs significantly by reserving number of instances for a year or for three years. Reservation is only a billing concept and thus you don’t have to designate any istances as reserved. If you reserve 4 m1.small instances, Amazon will make 4 of your running m1.small instances as reserved. For more information visit Werner Vogels’s (Amazon CTO) blog.

There might be numerous other ways you can save money. Please feel free to leave a comment to share your ways. The more the merrier!

RDS does not give you super privileges. That is why you won’t be able to change the global time zone by simply executing:

SET GLOBAL time_zone = 'US/Pacific';


The default_time_zone db parameter is not modifiable.

In this Amazon forum thread, you can clearly see the AWS guy saying that there is no way to change the time zone of a RDS instance.

It is very important to understand that unlike Java (or other programming languages), MySQL does not store time zone in its date, datetime or timestamp datatypes. That is why it’s important that all the dates are stored in one time zone. Most people prefer to store them in their own time zone for easier calculations.

Since you cannot change the database server settings, you will have to change your application. Here are some approaches you can take to make sure that the date and time stored in your database is in your desired time zone.

1) Set time zone per database connection
When ever you create a connection, set time zone using

SET time_zone = 'US/Pacific';

This time zone will be valid only for the connection. That is why it must be done for every single connection you open. If you have a java application, you can use a wrapper around the JDBC driver to accomplish this.

Please note that MySQL allows you to specify time zone by its name (such as ‘US/Pacific’) or by the difference in hours (such as ‘-8:00′). If you are in a part of United States or any other part of the world that uses day light savings, do not use the difference of hours option for obvious reasons.

Named time zones can only be used if the time zone related tables are populated in MySQL. Thankfully they seem to be populated in RDS.

2) Use convert_tz function
You can use MySQL’s convert_tz function to convert the time zone. For example:


insert into my_table (created_date) values (convert_tz(now(), 'GMT', 'US/Pacific'));


For more information on convert_tz function visit convert_tz documentation on MySQL website

3) Set the date/time pragmatically in your application code in every query.
Thus if you have created_date column in your table, do not rely on timestamp datatype’s default value. The default value will be in UTC time zone. Set the value explicitly using application code. If you are using a prepared statement this becomes very easy. All you have to make sure is to pass a date object with appropriate time zone set in it. In order for your application to calculate correct date, you have to make sure that your application instance’s time zone is set to your time zone.

Amazon ec2 instances allow you to change their default time zones. If your web server is also deployed in ec2, I would suggest you to bundle an AMI with your time zone and use it to launch all your instances. I think most of the ubuntu public AMIs have time zone set to UTC.

RDS Pros:
1) No need to maintain MySQL in the cloud. Amazon will maintain it. Amazon will apply all the patches and upgrade the server software.

2) It’s very easy to scale up. All you have to do is execute rds-modify-db-instance command. You can change your db class to the bigger/better machine.

3) Snapshot based backups are very easy to manage.

4) No need to manage an ec2 volume, backup and an ec2 instance separately to run your database in the cloud. RDS combines all the three in one package.

RDS Cons:
1) The biggest problem about RDS is the 4 hour per week maintenance window. I don’t believe that this maintenance takes place every week. They only take down your database when they are applying patches or upgrading the database server. They will take down your database in your specified time only. This time can be changed any time by simply executing an api call. I also believe that in most cases the downtime will be much less than 4 hours. Nevertheless the threat of downtime exists every week!

2) Does not expose my.cnf file of the database. It instead gives you an API to modify the parameters

3) If you are an advanced MySQL user and want to use tweaked MySQL (such as by Percona http://www.percona.com/), RDS is clearly not a solution.

4) As of now, there is no replication support. This means you cannot have any read only slaves. I have read somewhere that they are working on it.

Clearly RDS is not a silver bullet. In fact in many cases where your application cannot run without a database, you may not be able to afford the weekly downtime. I talked to some friends and the 4 hour weekly downtime was their the biggest problem with RDS. I wish that Amazon would declare the downtime whenever the need arises (such as when new MySQL patches are ready) and less frequently (such as once a month).

We did not have a full time system administrator in our team. We were bunch of software engineers trying to manage our ec2 instances by ourselves. Furthermore our application wasn’t dependant on database completely. Most of the data was cached. Thus we could afford an occasional downtime. That is why we chose RDS.

We have found a solution. Ylastic provides an amazing and inexpensive web based interface for various AWS services. It allows you to name your ec2 instances and remembers it. If you have an autoscaling setup it will automatically name your autoscaled instances when they start e.g. webserver001, webserver002 etc.

Ylastic does much more than just naming the servers. It has a very user friendly user interface and also provides an iphone website from which you can start, stop instances etc. It also has a concept of locking an instance so that you don’t accidentally terminate it.

ylastic ec2 instances table

It has nice graphs for cloudwatch data.

ylastic cloudwatch graphs

It supports most of the AWS services such as EC2, RDS, Auto scaling, S3, Simple DB.

ylastic dashboard

It also lets you schedule tasks such as taking a snapshot of your ebs volumes etc. It keep tabs on your instances by receiving alerts when the state of the launched instance changes.

Ylastic costs only $25 per month per Amazon account. It allows you to create multiple users for the same Amazon account. Furthermore ylastic guys are amazingly fast. Their customer service usually comes back with an answer within the first hour. We have hardly had any complaints though. Most of the times we have suggested new features and these people have implemented them in few days.

I totally recommend ylastic. We are very happy with it at Bedrock. It has made our life easier.

In pre RDS era, you would have fired up an instance, installed MySQL yourself on it, configured an EBS volume so that the database data storage is persistent. You would have setup a cron to snapshot your EBS volume. But RDS saves the headache of managing an EBS volume and your instance separately. It offers you installed MySQL with persistent storage. It also offers you automated database backup. And setting it up is as easy as 1,2,3…

Here is what you do to setup a database using RDS:

1) Sign up for RDS by visiting http://aws.amazon.com/rds.

2) Install RDS tools on your local machine so that you can execute RDS setup commands. You can download the toolkit from the download page.
Installation of the client toolkit involves the following procedure:

a) Unzip the downloaded file into a suitable folder, say /home/vaibhav/software/rds-tools

b) Make sure that you have java 1.5 or up installed on your machine and  JAVA_HOME variable is set. If not install the latest version of java and set JAVA_HOME to the folder where java is installed.

c) Set AWS_RDS_HOME=/home/vaibhav/software/rds-tools in your .bashrc or execute the following command to set AWS_RDS_HOME for this session

export AWS_RDS_HOME=/home/vaibhav/software/rds-tools

d)Add AWS_RDS_HOME to your path by executing export PATH=$PATH:$AWS_RDS_HOME/bin

e) Rename credential-file-path.template in AWS_RDS_HOME (/home/vaibhav/software/rds-tools) to any name of your choice say credentials.txt. Set AWS_CREDENTIAS_FILE variable by executing

export AWS_CREDENTIAL_FILE=<the path of the file created above - credentials.txt

3) Now let’s create a security group before creating a database instance. This is because we would like to supply this security group while creating an instance. A security group can be created by executing

rds-create-db-security-group --db-security-group-name database  --db-security-group-description 'Default mysql database security group'

This will create a security group called database. If you want to connect to the mysql server from your local machine, you will need to add authorization to this group by executing

rds-authorize-db-security-group-ingress --db-security-group-name database -i 71.168.56.100/32

Here 71.168.56.100/32 is the ip address range of the network from which you want to access the database and 123456789012 is your Amazon AWS account number

4) Now create the database instance. The database instance can be created by executing the following command:

rds-create-db-instance --db-instance-identifier reporting --allocated-storage 5 --db-instance-class db.m1.small --engine MySQL5.1 --master-username master --master-user-password s0mePassw0rd --db-name MyFirstDb --availability-zone us-east-1a --preferred-maintenance-window Sun:12:00-Sun:16:00 --preferred-backup-window 10:00-12:00 --db-security-groups database --headers


There are few parameters that you need to understand very well before executing this command. The preferred-maintenance-window is the 4 hour time period Amazon requires to apply patches and to carry out other maintenance tasks on the database server. preferred-backup-window is the daily two hour window in which your database will be backed up. The database won’t be down in the period, but its performance could get affected because of the backup operation. Please note that both the time periods must be specified in UTC time. If you are going to access this database from an Amazon ec2 instance then choose an availability zone that is the same as the web server so that you are not charged for the bandwidth. You should change other parameters such as the database instance type, allocated storage as per your requirement. More information on these parameters can be obtained from the CLI reference page. The instance name will be reporting as specified. To find out its address, you can simply execute

rds-describe-db-instances

This command will list all the db instances with their dns names. The dns name will be something like reporting.crly0gpnhroz.us-east-1.rds.amazonaws.com. The port will also be in the output (3306). You can use this dns name and port to connect to the database using any mysql client.