Warning: This document describes an old release. Check here for the current version.

EC2 Client Notes

This section explains how to use 3rd-party EC2 and S3 clients with the Nimbus EC2 frontends, both SOAP and Query.

• Uploading VM Images to Cumulus with Boto

• Uploading VM Images to Cumulus with s3cmd

• Using the EC2 Query frontend from Python with Boto

• A Full Example with Boto

• Using the EC2 SOAP frontend from the console

Uploading VM Images to Cumulus with Boto (#)

The python library boto can be used to upload images to Cumulus. Once a VM is staged into Cumulus it can be submitted for execution.

In order for Nimbus to efficiently find VM images in Cumulus a naming convention is used. Three pieces of information are needed to follow this convention:

1. The repository bucket name
By default this is Repo
2. The image name prefix
By default this is VMS
3. Your canonical ID
The site admin must provide this to you at the time of your account creation. It can also be found in your cloud clients configuration file (More Info)

That information is used to form a Cumulus url in the following way:

cumulus://<cumulus hostname>/<bucket name>/<prefix>/<canonical ID>/<image name>

Once you have the proper url formed you can use that information and boto to upload the image into the storage repository.

from boto.s3.key import Key
from boto.s3.connection import OrdinaryCallingFormat
from boto.s3.connection import S3Connection

s3id="<Cumulus ID>"
s3pw="<Cumulus password>"
host="<Cumulus hostname>"
port=<Cumulus port>
canonical_id="<Canonical ID>"

cf = OrdinaryCallingFormat()
s3conn = S3Connection(s3id, s3pw, host=host, port=port, is_secure=False, calling_format=cf)

bucket = s3conn.get_bucket("Repo")
k = Key(bucket)
k.key = "VMS/" + canonical_id + "/" + image_name
k.set_contents_from_filename(path_to_image)

At this point your image will be ready for submission using the name image_name

Uploading VM Images to Cumulus with s3cmd (#)

There is more information here.

Using the EC2 Query frontend from Python with Boto (#)

Nimbus 2.5 supports for the EC2 Query interface, which is used by the excellent Python client, boto. In order to use this interface, you must have Query credentials from the cloud administrator. These are composed of an access identifier (usually the hash of your DN) and a shared secret key.

You also need the URL of the Query interface on the Nimbus service node. See the University of Chicago cloud for example.

Before you can launch a VM you must first upload it to Cumulus. That process is described here

import boto
from boto.ec2.regioninfo import RegionInfo

# first create a region object and connection
region = RegionInfo(name="nimbus", endpoint="service.hostname.com")
conn =  boto.connect_ec2("YOUR_ACCESS_ID_HERE", "YOUR_SECRET_KEY_HERE",
                         port=SERVICE_PORT_NUMBER, region=region)

# then do something with the connection
conn.run_instances("image_name")

Please refer to the boto documentation for more details on usage.

A Full Example with Boto (#)

import boto
from boto.s3.key import Key
import time
from boto.s3.connection import OrdinaryCallingFormat
from boto.s3.connection import S3Connection
from boto.ec2.connection import EC2Connection
from boto.ec2.regioninfo import RegionInfo
    
access_id="<Cumulus ID>"
access_secret="<Cumulus password>"
host="<Cumulus hostname>"
port=<Service port: 8444>
cumulusport=<Cumulus port: 8888>
canonical_id="<Canonical ID>"

region = RegionInfo(name="nimbus", endpoint=host)
ec2conn = boto.connect_ec2(access_id, access_secret, region=region, port=port)
cf = OrdinaryCallingFormat()
s3conn = S3Connection(access_id, access_secret, host=host, port=cumulusport, is_secure=False, calling_format=cf)

# upload the file according to the Nimbus naming convention
bucket = s3conn.get_bucket("Repo")
k = boto.s3.key.Key(bucket)
k.key = "VMS/%s/%s" % (canonical_id, "my_image_name")
k.set_contents_from_filename(<path to image file>)

# now run the VM
reservation = ec2conn.run_instances("my_image_name")
instance = reservation.instances[0]

# poll for status
while instance.state != 'running':
    print 'instance is %s' % instance.state
    time.sleep(30)
    instance.update()

# terminate the instance
reservation.instances[0].terminate()

Using the EC2 SOAP frontend from the console (#)

When using a cloud running the EC2 frontend, you can download this EC2 client from Amazon or try a number of different client that are out there.

• Nimbus 2.3+ supports the "2009-08-15" WSDL (this EC2 client) .
• Nimbus TP2.2 supports the "2008-05-05" WSDL (this EC2 client).
• The older Nimbus TP2.0 supports the "2008-02-01" WSDL (this EC2 client).
• So the default client may not always be the one to use. See a specific cloud's documentation for the definitive tools URL (for example, the Nimbus cloud). And see enhancement 6558.

With the client from Amazon, you need to adjust the following environment variables (using other clients, the configurations can vary).

• EC2_HOME - Set the base directory (such that $EC2_HOME/bin is valid)
• EC2_URL - Set the service endpoint to the Nimbus based service
• EC2_CERT - Set the public credential to your public .pem based cert
• EC2_PRIVATE_KEY - Set the private credential to an unencrypted .pem based key

Example:

The URL for EC2_URL will be provided to you by an administrator or some documentation (HOST:PORT in the example is a placeholder for an actual hostname and port number)

For example, see the Nimbus cloud docs for running EC2 clients.

If you have a problem, consult the troubleshooting guide.

Before being able to launch an instance, you will need to register a keypair like so:

• $ MYPUBKEY=`cat .ssh/id_rsa.pub`

• Check to make sure those contents look OK:

  $ echo $MYPUBKEY

• Send the public key value to the service, labelling it "mykey"

  $ ec2-add-keypair "mykey||$MYPUBKEY"

List the available images in your personal directory:

Pick one, for example "hello-cloud". And with the "-k" argument, use the ssh key label you used above to register an ssh public key.

Check status:

After seeing 'pending' change to 'running', log in at the address printed:

Terminate using the "instance ID" printed when you first launched (and also available from ec2-describe-instances). It looks like "i-4662834e":