For instance, let's assume that your corporate firewall has the IP address 1.2.3.4 and your
corporation has been assigned that network as a /20, ie 1.2.3.0/20 (1.2.3.0-1.2.18.255) you could
easily restrict ssh access to hosts only on your corporate network thus:

PROMPT>  ec2-authorize default -p 22 -s 1.2.3.0/20
PERMISSION     default  ALLOWS  tcp     22      22      FROM    CIDR   1.2.3.0/20

PROMPT> ec2-revoke default -p 22 -s 1.2.3.0/20

Disable Password-based Login

# vi /etc/ssh/sshd_config

PasswordAuthentication yes

PasswordAuthentication no

Key Generation and Distribution

In the example below (to be run on your local machine, not your EC2 instance) replace "user" with the
actual user's login, name or some other unique identifier.

ssh-keygen -b 1024 -f user -t dsa

Then I attached them to my EC2 instance:

Then I switched over to my instance, formatted and mounted them, and I was all set:

Here is some additional coverage:

• Amazon's Werner  Vogels talks about  this feature in Persistent Storage for Amazon EC2.



• RightScale's Thorsten vok Eiken does too, in Amazon takes EC2 to the next level with persistent storage volumes.

FUSE Interfaces

s3fs (Open Source)

s3fs is a FUSE filesystem that allows you to mount an Amazon S3 bucket as a local filesystem. It stores files natively and transparently in S3 (i.e., you can use other programs to access the same files). Maximum file size is 5G.

PersistentFS (Closed Source, Current Development Version is Free)

PersistentFS is a fast and efficient POSIX-compliant file system that provides unlimited online storage in the Amazon Web Services (AWS) storage cloud.

InfiniteFTP (Closed Source, Paid Service) over CurlFtpFs (Open Source)

InfiniteFTP provides an FTP interface to Amazon S3 - while this isn't a file system approach then you could add a FUSE layer over the top to make it appear as one.

CurlFtpFS is a filesystem for accessing FTP hosts based on FUSE and libcurl.

NBD Interfaces

ElasticDrive (Closed Source, Paid Service)

ElasticDrive is a Distributed Remote Storage Application that allows you to access your data regardless of location with the assurance your data is safe. It makes it possible for a remote storage resource, such as Amazon's Simple Storage Service (S3), Nirvanix & Xdrive to behave like a local hard drive.

Cost Considerations

I've only considered the options I could find for storage backed by S3 - my main in head reason for this was to avoid transfer charges. There will still be charges for the GET, PUT and LIST requests so I also need to consider the cost implications of using S3 to assist with persistent storage.

Hardware: Monitoring Your CPU

Your first consideration in determining what to monitor is to figure out whether monitoring is really a good idea. If you're doing CPU-intensive work on your system (that is, a lot of calculation work), then this is a good item to monitor. Obviously, if the system seems to crash a lot or it runs very slowly, CPU is definitely on the short list of items to monitor.

One of the nice things about Linux is that it has a plethora of tools available for monitoring. Using vmstat, look at the run queue time (kthr - r column). Generally, the larger this number, the busier the CPU is. You can capture this number to a variable or store it on disk, possibly creating an XML file that you can parse and compare values. The following code shows an example of how to implement CPU monitoring.

require 'EC2'

require 'thread'

# this function runs vmstat and parses the output to determine if the CPU is overloaded
def get_cpu_load
  cpu_load = 0

  # run vmstat measurement 6 times with a delay of 10 seconds between measurements
  IO.popen("vmstat -n 10 6") { |io|
    # the first line of input is the header; we can discard it
    io.gets
    
    6.downto(1) {
      # get an array of all integers returned by vmstat
      stats = io.gets.split

      # sum together the run queue load
      cpu_load += stats[0].to_i
    }
  }
  cpu_load
end

access_key_id = ENV['AMAZON_ACCESS_KEY_ID']

secret_key = ENV['AMAZON_SECRET_ACCESS_KEY']

instance_id = ENV['AMAZON_INSTANCE_ID']

max_instances = 2
max_instances |= ENV['AMAZON_MAX_INSTANCES']

#create the instance of the ec2 object

ec2 = EC2::Base.new( :access_key_id => accessid_key_id, :secret_access_key => secret_key )

#run the instance

ec2.run_instances(:image_id => 'ami-f9907590')

#periodically check CPU load with vmstat

th = Thread.new do

  logit = File.open('avg_cpu_load.xml', 'w')

  loop do
    
    cpu_load = get_cpu_load

    #log CPU performance
    logit.puts("<vmstat><time>#{Time.now}</time><avg_load>#{cpu_load}</avg_load></vmstat>\n")

    # let's assume 9 is too much
    if (cpu_load => 9)

      logit.puts("<vmstat><time>#{Time.now}</time><message>maximum CPU load exceeded</message></vmstat>\n")

      #start more instances
      ec2.run_instances(:image_id => 'ami-f9907590', 
      :min_count => 2, :max_count => max_instances)

    end

    # try again in 5 minutes
    sleep(60 * 5)
  
  end

  logit.close

end

Hardware: Monitoring Memory

Is monitoring memory a good idea? Because a system short on memory for running
applications can obviously cause performance issues, and because this is easy to check, this is a great candidate.

Memory is easy to check. On Linux you can simply use the free command and use a regular expression to pipe the output you want to a variable (or store it in an XML file, if you want it to persist between program runs). The following code example shows a way to monitor memory.

require 'EC2'

require 'thread'

# this function runs free and parses the output to determine if the memory is overloaded
def get_free_memory
  free_memory = 0

  # run free measurement 6 times with a delay of 10 seconds between measurements
  6.downto(1) {
    IO.popen("free -m") { |io|
      # we can discard the first two lines of output
      io.gets; io.gets
    
      # get an array of all tokens returned by free
      stats = io.gets.split

      # we're only interested in the fourth item in the array
      free_memory += stats[3].to_i
    }
    sleep(10)
  }
  free_memory / 6 # return the average free memory available for all 6 measurements
end

access_key_id = ENV['AMAZON_ACCESS_KEY_ID']

secret_key = ENV['AMAZON_SECRET_ACCESS_KEY']

max_instances = 2
max_instances |= ENV['AMAZON_MAX_INSTANCES']

#create the instance of the ec2 object

ec2 = EC2::Base.new( :access_key_id => accessid_key_id, :secret_access_key => secret_key )

#run the instance

ec2.run_instances(:image_id => 'ami-f9907590')

#periodically check memory usage with free

th = Thread.new do

  logit = File.open('avg_free_memory.xml', 'w')

  loop do
    
    free_memory = get_free_memory

    #log free memory available
    logit.puts("<free><time>#{Time.now}</time><avg_free_mem>#{free_memory}</avg_free_mem></free>\n")

    # let's assume 64 megabytes is too little available memory
    if (free_memory <= 64)

      logit.puts("<free><time>#{Time.now}</time><message>maximum memory load exceeded</message></free>\n")

      #start more instances
      ec2.run_instances(:image_id => 'ami-f9907590', 
      :min_count => 2, :max_count => max_instances)

    end

    # try again in 5 minutes
    sleep(60 * 5)

  end

  logit.close

end

Hardware: Monitoring Disk I/O

Is monitoring disk input/output (I/O) a good idea? The answer in this case depends on whether you are actually experiencing an I/O problem. If you decide you need to monitor I/O, it would be reasonable to use iostat to monitor for the iowait parameter--the time the CPU is waiting while it has no pending I/O request. If this number is high, it's an indication there is an I/O bottleneck. As in the hardware monitoring examples discussed in previous sections, this number can be extracted using regular expressions and either
be assigned to a variable, or to output to XML for later processing.

However, it's important to realize that iostat is a tool that should not be run too often. Because it has a high overhead, iostat can contribute to the very problem for which it is monitoring. So, monitor your I/O if you suspect a problem, but don't set it up for continuous monitoring.

Configuration: Apache and the J2EE Server in an EC2 Environment

There are many ways to connect Apache with Tomcat. Over the past few years, native Tomcat connectors
have quickly superseded each other and there have been options to connect by using the generic HTTP protocol.
You will find protocol names such as mod_jk, mod_jk2, mod_ajp, mod_proxy, mod_rewrite, and
other variations. Many online help texts are giving outdated or contradictory advice, making it hard to choose the proper connection protocol and settings.

Having tried most of the available connection choices, I know that each connection protocol has its challenges. It turns out
that the option that's the simplest and
the most consistent with other similar Apache features is the newest option: mod_proxy_ajp. The mod_proxy_ajp
protocol is similar to mod_proxy_http except that it saves you from providing ProxyPassReverse
lines in addition to ProxyPass, which means the AJP protocol is easier to maintain than the HTTP protocol.

Note:
See http://developer.amazonwebservices.com/connect/entry.jspa?entryID=1015 for an
Amazon Machine Image (AMI) with Apache and Tomcat preinstalled.

Before setting up Apache, you should start at the bottom: configure Tomcat's [TOMCAT_HOME]/conf/server.xml
file. For Tomcat versions up to 5.5, it is wise to start with the example content from server-minimal.xml. Tomcat 6.0 doesn't provide this example, but the content of
the default server.xml file is fairly minimal in 6.0. Find the AJP connector declaration:

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

Then, decide whether port 8009 is suitable for your situation. Let's assume you are running only one Tomcat per server instance. In
some situations there are reasons to do otherwise, but they'd complicate this explanation.

Now, it's time to change the Apache configuration, [APACHE_HOME]/conf/httpd.conf. First, find the lines
beginning with LoadModule and uncomment them:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_http_module modules/mod_proxy_ajp.so

Then, at the bottom, add the following:

<Proxy balancer://mycluster>
	Order deny,allow
	Allow from all
	
		BalancerMember ajp://[SERVER1].amazon.com:8009/myjavaapp
	BalancerMember ajp://[SERVER2].amazon.com:8009/myjavaapp
	#... as many as you need here ...</Proxy>	

ProxyPass /myjavaapp balancer://mycluster

The last configuration detail is the CNAME in your DNS settings. Who your
provider is will determine how you set these. Many larger application service providers give you your own dashboard where you can create A records, MX records, and CNAMES. Please refer to the documentation
supplied by your provider for details.

Wowza Media Server Pro On Amazon EC2

Running JEE applications on Amazon EC2: deploying to 20 machines with no money down

class ClusterTest extends GroovyTestCase {

  void testSomething() {

    AWSProperties awsProperties = new 

        AWSProperties("/…/aws.properties")

    def ec2 = new EC2(awsProperties)

    def explodedWar = '…/projecttrack/webapp/target/ptrack'

    ClusterSpec clusterSpec = 

       new ClusterSpec()

            .schema("ptrack", ["ptrack": "ptrack"], 

                    ["src/test/resources/testdml1.sql", 

                     "src/test/resources/testdml2.sql"])

            .slaves(1)                  

            .tomcats(1)

            .webApp(explodedWar, "ptrack")

            .catalinaOptsBuilder({builder, databasePrivateDnsName ->

                 builder.arg("-Xmx500m")

                 builder.prop("com.sun.management.jmxremote")

                 builder.prop("com.sun.management.jmxremote.port", 8091)

                 builder.prop("com.sun.management.jmxremote.authenticate",             

                                     false)

                 builder.prop("com.sun.management.jmxremote.ssl", false)

                 builder.prop("ptrack.application.environment", "ec2")

                 builder.prop("log4j.configuration", 

                               "log4j-minimal.properties")

                 builder.prop("jdbc.db.server", databasePrivateDnsName)})

    SimpleCluster cluster = new SimpleCluster(ec2, clusterSpec)

    cluster.loadTest("…/projecttrack/functionalTests/jmeter/SimpleTest.jmx",

        [1, 2])

    cluster.stop()

  }

}

Running JEE applications on Amazon EC2: deploying to 20 machines with no money down
>

class ClusterTest extends GroovyTestCase {
>

  
void testSomething() {
>

    
AWSProperties awsProperties = new
> 

        
AWSProperties("/…/aws.properties")
>

    
def ec2 = new EC2(awsProperties)
>

    
def explodedWar = '…/projecttrack/webapp/target/ptrack'
>

    
ClusterSpec clusterSpec =
> 

       
new ClusterSpec()
>

            
.schema("ptrack", ["ptrack": "ptrack"],
> 

                    
["src/test/resources/testdml1.sql",
> 

                     
"src/test/resources/testdml2.sql"])
>

            
.slaves(1)
>                  

            
.tomcats(1)
>

            
.webApp(explodedWar, "ptrack")
>

            
.catalinaOptsBuilder({builder, databasePrivateDnsName ->
>

                 
builder.arg("-Xmx500m")
>

                 
builder.prop("com.sun.management.jmxremote")
>

                 
builder.prop("com.sun.management.jmxremote.port", 8091)
>

                 
builder.prop("com.sun.management.jmxremote.authenticate",
>             

                                     
false)
>

                 
builder.prop("com.sun.management.jmxremote.ssl", false)
>

                 
builder.prop("ptrack.application.environment", "ec2")
>

                 
builder.prop("log4j.configuration",
> 

                               
"log4j-minimal.properties")
>

                 
builder.prop("jdbc.db.server", databasePrivateDnsName)})
>

    
SimpleCluster cluster = new SimpleCluster(ec2, clusterSpec)
>

    
cluster.loadTest("…/projecttrack/functionalTests/jmeter/SimpleTest.jmx",
>

        
[1, 2])
>

    
cluster.stop()
>

  
}
>

}
>