User:Gholms/EC2 Primer

From FedoraProject

< User:Gholms(Difference between revisions)
Jump to: navigation, search
(Choose a Region)
(Refactored)
Line 1: Line 1:
[http://aws.amazon.com/ '''Amazon Web Services'''] (AWS) comprise a '''public cloud''', a collection of computing services that allows one to run services and virtual machines over the Internet on Amazon's computer infrastructure.  Fedora publishes system images for AWS's virtual machine platform, '''Amazon Elastic Compute Cloud''' (EC2), which allows people to create Fedora virtual machines in the cloud with very little effort.  The objective of this primer is to familiarize the reader with EC2's terminology and functionality.  For more detailed documentation, see the AWS website.
+
[http://aws.amazon.com/ '''Amazon Web Services'''] (AWS) comprise a '''public cloud''', a collection of computing services that allows one to build and run software services in Amazon's data centers.  Fedora publishes system images for AWS's virtual machine platform, '''Amazon Elastic Compute Cloud''' (EC2), which allows one to create virtual machines in the cloud with very little effort.  The objective of this primer is to familiarize the reader with EC2's terminology and functionality.  For more detailed documentation, see the AWS website.
  
 
== EC2 Concepts ==
 
== EC2 Concepts ==
Line 5: Line 5:
 
What follows are some short explanations of EC2 terminology.  For more detailed information, see the EC2 documentation.
 
What follows are some short explanations of EC2 terminology.  For more detailed information, see the EC2 documentation.
  
=== Accounts ===
+
=== Images and Instances ===
  
To use AWS you need to create an online '''account'''.  You can do this by going to the [http://aws.amazon.com/ AWS web site], clicking on "Create an AWS Account", and following the instructions.
+
A '''machine image''' is a snapshot of a system (specifically its <code>/</code> filesystem) that provides the basis for a virtual machine in EC2.  When running a new virtual machine in EC2 you choose a machine image to use as a template.  The new virtual machine is then an '''instance''' of that machine image that contains its own copy of everything in the imageThe instance keeps running until you stop or terminate it, or until it fails.  If an instance fails, you can launch a new one from the same image. You can create multiple instances of a single machine image. Each instance will be independent of the others.
  
AWS is designed as a pay-as-you-go online serviceMuch of EC2 is free for new users; the rest is available for per-hour or per-month fees that are detailed on the [http://aws.amazon.com/ec2/#pricing EC2 Website]As such, Amazon requests a credit card number to keep on file with your new account.
+
You can use a single image or multiple images, depending on your needsFrom a single image, you can launch different types of instancesAn '''instance type''' defines what hardware the instance has, including the amount of memory, disk space, and CPU power.
  
{{admon/tip|Resources for test days|The [[Cloud SIG|Fedora Cloud SIG]] may provide sponsored accounts or financial reimbursement for EC2 test days.}}
+
Amazon, Fedora, other groups, and individuals publish images for public use.  You might only need to use images that reputable sources provide, and you can simply customize the resulting instances to suit your needs as you launch them.  You can also create your own machine images.
 +
 
 +
Machine images in EC2 are sometimes referred to as '''AMIs'''.
 +
 
 +
Machine images have identifiers that begin with <code>ami</code>, such as <code>ami-6ebe4507</code>.  Instances have identifiers that begin with the letter <code>i</code>, such as <code>i-12459dbd</code>.
  
 
=== Regions and Availability Zones ===
 
=== Regions and Availability Zones ===
Line 17: Line 21:
 
Amazon hosts datacenters many parts of the world.  Those from a particular part of the world make up a '''region'''.  Regions' names are based on their locations, such as in <code>us-east-1</code>.
 
Amazon hosts datacenters many parts of the world.  Those from a particular part of the world make up a '''region'''.  Regions' names are based on their locations, such as in <code>us-east-1</code>.
  
Regions are broken into '''availability zones'''.  Distributing a web application amongst several availability zones can help improve its reliability if an availability zone has problems.  Availability zones' names are based on the regions in which they reside, such as <code>us-east-1a</code>.
+
Regions are broken up into '''availability zones''', which are designed to isolate failures from one another but still provide faster communication than communication between regions.  Distributing a web application amongst several availability zones can help improve its reliability if an availability zone encounters problems.  Availability zones' names are based on the regions in which they reside, such as <code>us-east-1a</code>.
  
=== Images and Instances ===
+
=== Storage ===
  
A machine '''image''' is a snapshot of a Fedora installation that provides the basis for a virtual machine in EC2.  When running a new virtual machine in EC2 you choose a machine image to use as a template.  The new virtual machine is then an '''instance''' of that machine image that contains its own copy of everything in the image.  This copy, called '''instance storage''', lasts for the lifetime of the instance and is destroyed when the instance is terminated (i.e. destroyed).
+
EC2 instances use one or more of three types of storage provided by AWS:
  
You can create multiple instances of a single image.  Each instance will be independent of the others.
+
==== Amazon S3 ====
  
Machine images have identifiers that begin with the letters <code>ami</code>, such as <code>ami-6ebe4507</code>Instances have identifiers that begin with the letter <code>i</code>, such as <code>i-12459dbd</code>.
+
Amazon Simple Storage Service (S3) is a web service-based storage system that is accessible inside EC2 and elsewhere on the InternetAs this primer will not focus on S3, see the [http://aws.amazon.com/s3 Amazon S3] documentation.
  
Machine images in EC2 are sometimes referred to as '''AMIs'''.
+
==== Elastic Block Store (EBS) ====
  
=== Security Groups ===
+
Amazon Elastic Block Store (EBS) provides instances with persistent, disk-like storage that you can attach to and detach from instances, similar to portable disk drives.  By creating EBS '''volumes''' and attaching them to instances you can store data that you wish to be portable to more than one instance in the event an instance fails or is replaced.  Since instances' root filesystem tend to have limited space, volumes also provide a simple way of adding additional disk capacity to instances.
  
A '''security group''' defines firewall rules for your instances.  These rules specify which incoming network traffic should be delivered to an instance (e.g., accept web traffic on port 80 or SSH traffic on port 22).  All other traffic is ignored.  You can modify the rules for a group at any time.
+
Volumes have identifiers that begin with <code>vol</code>, such as <code>vol-ffe93704</code>.
  
Every instance runs inside of a security groupYou can create your own security groups, or you can use the <code>default</code> security group that EC2 provides for you.  When you run a new instance it will run in the <code>default</code> security group unless you choose a different one.
+
You can create a backup '''snapshot''' of a volumeFrom the snapshot you can then create a new volume and attach it to another instance.
  
=== Elastic Block Storage (EBS) ===
+
Snapshots have identifiers that begin with <code>snap</code>, such as <code>snap-773491a0</code>.
  
==== Volumes ====
+
==== Instance Storage ====
  
A typical instance does not have a large amount of disk space.  In addition, instance storage does not persist past the lifetime of an instance.  Elastic block '''volumes''' are designed to solve this problem by allowing one to request disk space of arbitrary sizes and then '''attach''' them to instances.  Think of a volume as a disk that you can attach and detach from instances just as one would attach and detach a USB drive from regular computers.
+
Some instance types have '''instance storage''', scratch space that persists only as long as an instance runsInstance storage is destroyed when an instance stops, terminates, or failsFor this reason, it is also referred to as ephemeral storage.
  
An EBS volume is not normally destroyed when the instance to which it is attached terminatesFor this reason, EBS volumes are appropriate for storing important data or data that may need to move from one instance to another.
+
When EC2 was first introduced, all machine images were ''backed by instance storage'', meaning that their instances' root filesystems were stored in instance storageMachine images can now also be ''backed by EBS'', meaning that their instances' root filesystem instead reside on EBS volumes.
  
EBS volumes can also be quite large, so it is wise to put any data of significant size on them to prevent instances' limited instance storage from filling up.
+
=== Security Groups ===
  
EBS volumes have identifiers that begin with <code>vol</code>, such as <code>vol-ffe93704</code>.
+
A '''security group''' defines firewall rules for your instances.  These rules specify which incoming network traffic should be delivered to an instance (e.g., accept web traffic on port 80 or SSH traffic on port 22).  All other traffic is ignored.  You can modify the rules for a group at any time.
  
==== Snapshots ====
+
Every instance runs inside of a security groupYou can create your own security groups, or you can use the <code>default</code> security group that EC2 provides for you.  When you run a new instance it will run in the <code>default</code> security group unless you choose a different one.
 
+
Volume '''snapshots''' allow one to create backup copies of EBS volumesThese snapshots can then be used to re-create volumes.  Snapshots provide a convenient way of making backups of important data in the cloud.  They are also a convenient way to make an independent copy of a set of data for a number of instances.
+
 
+
EBS snapshots have identifiers that begin with <code>snap</code>, such as <code>snap-773491a0</code>.
+
 
+
=== Keypairs ===
+
 
+
A '''keypair''' is a pair of SSH keys that you can use to log into EC2 instances after they are running, just as you can use them to log into regular computersEC2 makes the ''public'' key from a keypair available to instances so you can use the ''private'' key to log in with <code>ssh</code>.
+
  
 
== Getting Started with Fedora on EC2 ==
 
== Getting Started with Fedora on EC2 ==
Line 61: Line 57:
 
=== Getting account details ===
 
=== Getting account details ===
  
You can begin using Fedora on EC2 once you have an EC2 account.  If you do not yet have one, go to the [http://aws.amazon.com/ AWS web site] to create one.
+
To use AWS you need to create an online account.  You can do this by going to the [http://aws.amazon.com/ AWS web site], clicking on "Create an AWS Account", and following the instructions.
 +
 
 +
{{admon/note|Amazon AWS is not free|AWS is designed as a pay-as-you-go online service.  Much of EC2 is free for new users; the rest is available for per-hour or per-month fees that are detailed on the [http://aws.amazon.com/ec2/#pricing EC2 Website].  As such, Amazon requests a credit card number to keep on file with your new account.  If you are participating in a Fedora Test Day, the [[Cloud SIG|Fedora Cloud SIG]] may be able to provide you with a sponsored account or financial reimbursement.}}
  
One can interact with EC2 through either a web-based [https://console.aws.amazon.com/ec2/ management console] or via '''euca2ools''', a suite of command line tools designed for services like EC2.  This document focuses on the command line tools.
+
One can interact with EC2 through either a web-based [https://console.aws.amazon.com/ec2/ management console] or via '''euca2ools''', a suite of command line tools designed for services like EC2.  This tutorial will focus on using EC2 with euca2ools at the command line.
  
 
To using the command line tools you first need to obtain access keys for your account.  You can find them by going to the AWS management console on the web, clicking your name on the top, followed by  
 
To using the command line tools you first need to obtain access keys for your account.  You can find them by going to the AWS management console on the web, clicking your name on the top, followed by  

Revision as of 03:48, 12 October 2011

Amazon Web Services (AWS) comprise a public cloud, a collection of computing services that allows one to build and run software services in Amazon's data centers. Fedora publishes system images for AWS's virtual machine platform, Amazon Elastic Compute Cloud (EC2), which allows one to create virtual machines in the cloud with very little effort. The objective of this primer is to familiarize the reader with EC2's terminology and functionality. For more detailed documentation, see the AWS website.

Contents

EC2 Concepts

What follows are some short explanations of EC2 terminology. For more detailed information, see the EC2 documentation.

Images and Instances

A machine image is a snapshot of a system (specifically its / filesystem) that provides the basis for a virtual machine in EC2. When running a new virtual machine in EC2 you choose a machine image to use as a template. The new virtual machine is then an instance of that machine image that contains its own copy of everything in the image. The instance keeps running until you stop or terminate it, or until it fails. If an instance fails, you can launch a new one from the same image. You can create multiple instances of a single machine image. Each instance will be independent of the others.

You can use a single image or multiple images, depending on your needs. From a single image, you can launch different types of instances. An instance type defines what hardware the instance has, including the amount of memory, disk space, and CPU power.

Amazon, Fedora, other groups, and individuals publish images for public use. You might only need to use images that reputable sources provide, and you can simply customize the resulting instances to suit your needs as you launch them. You can also create your own machine images.

Machine images in EC2 are sometimes referred to as AMIs.

Machine images have identifiers that begin with ami, such as ami-6ebe4507. Instances have identifiers that begin with the letter i, such as i-12459dbd.

Regions and Availability Zones

Amazon hosts datacenters many parts of the world. Those from a particular part of the world make up a region. Regions' names are based on their locations, such as in us-east-1.

Regions are broken up into availability zones, which are designed to isolate failures from one another but still provide faster communication than communication between regions. Distributing a web application amongst several availability zones can help improve its reliability if an availability zone encounters problems. Availability zones' names are based on the regions in which they reside, such as us-east-1a.

Storage

EC2 instances use one or more of three types of storage provided by AWS:

Amazon S3

Amazon Simple Storage Service (S3) is a web service-based storage system that is accessible inside EC2 and elsewhere on the Internet. As this primer will not focus on S3, see the Amazon S3 documentation.

Elastic Block Store (EBS)

Amazon Elastic Block Store (EBS) provides instances with persistent, disk-like storage that you can attach to and detach from instances, similar to portable disk drives. By creating EBS volumes and attaching them to instances you can store data that you wish to be portable to more than one instance in the event an instance fails or is replaced. Since instances' root filesystem tend to have limited space, volumes also provide a simple way of adding additional disk capacity to instances.

Volumes have identifiers that begin with vol, such as vol-ffe93704.

You can create a backup snapshot of a volume. From the snapshot you can then create a new volume and attach it to another instance.

Snapshots have identifiers that begin with snap, such as snap-773491a0.

Instance Storage

Some instance types have instance storage, scratch space that persists only as long as an instance runs. Instance storage is destroyed when an instance stops, terminates, or fails. For this reason, it is also referred to as ephemeral storage.

When EC2 was first introduced, all machine images were backed by instance storage, meaning that their instances' root filesystems were stored in instance storage. Machine images can now also be backed by EBS, meaning that their instances' root filesystem instead reside on EBS volumes.

Security Groups

A security group defines firewall rules for your instances. These rules specify which incoming network traffic should be delivered to an instance (e.g., accept web traffic on port 80 or SSH traffic on port 22). All other traffic is ignored. You can modify the rules for a group at any time.

Every instance runs inside of a security group. You can create your own security groups, or you can use the default security group that EC2 provides for you. When you run a new instance it will run in the default security group unless you choose a different one.

Getting Started with Fedora on EC2

Getting account details

To use AWS you need to create an online account. You can do this by going to the AWS web site, clicking on "Create an AWS Account", and following the instructions.

Note.png
Amazon AWS is not free
AWS is designed as a pay-as-you-go online service. Much of EC2 is free for new users; the rest is available for per-hour or per-month fees that are detailed on the EC2 Website. As such, Amazon requests a credit card number to keep on file with your new account. If you are participating in a Fedora Test Day, the Fedora Cloud SIG may be able to provide you with a sponsored account or financial reimbursement.

One can interact with EC2 through either a web-based management console or via euca2ools, a suite of command line tools designed for services like EC2. This tutorial will focus on using EC2 with euca2ools at the command line.

To using the command line tools you first need to obtain access keys for your account. You can find them by going to the AWS management console on the web, clicking your name on the top, followed by Security Credentials, and scrolling down to the section titled Access Credentials. Make note of the Access Key ID and the Secret Access Key that appears beside it. Both of them should be long sets of alphanumeric characters. Create a file called .iamrc in your home directory that contains those keys in this format:

AWSAccessKeyId=your_access_key_id
AWSSecretKey=your_secret_key

Since euca2ools is designed to work with all AWS-compatible clouds, not just AWS itself, it needs to know which cloud to contact. Create a file called .eucarc in your home directory with the following content to point it toward AWS:

export AWS_CREDENTIAL_FILE=~/.iamrc
export EC2_URL=https://ec2.amazonaws.com/
export S3_URL=https://s3.amazonaws.com/
export EUARE_URL=https://iam.amazonaws.com/

source "$AWS_CREDENTIAL_FILE"
export EC2_ACCESS_KEY=$AWSAccessKeyId
export EC2_SECRET_KEY=$AWSSecretKey
export AWS_ACCESS_KEY=$AWSAccessKeyId
export AWS_SECRET_ACCESS_KEY=$AWSSecretKey
Finally, add these settings to your shell's environment by running:
source ~/.eucarc

Initial Setup

Install the Command Line Tools

Install the euca2ools package. To do so with yum, run:
yum install euca2ools

Choose a Region

Choose an EC2 region to use. Things to consider when choosing a region include how geographically close it is to you, the pricing for instances in that region, and whether the image you wish to use is available in that region. You can get a list of regions by running euca-describe-regions, which results in a list such as this:

REGION  eu-west-1       ec2.eu-west-1.amazonaws.com
REGION  us-east-1       ec2.us-east-1.amazonaws.com
REGION  ap-northeast-1  ec2.ap-northeast-1.amazonaws.com
REGION  us-west-1       ec2.us-west-1.amazonaws.com
REGION  ap-southeast-1  ec2.ap-southeast-1.amazonaws.com
When you choose an EC2 region you can make euca2ools start using it by editing the line that contains EC2_URL in your .eucarc file:
export EC2_URL=https://ec2.us-east-1.amazonaws.com/
...and then re-set the settings in your shell's environment:
source ~/.eucarc

Create a Keypair

Set up a Security Group

Run an Instance

Choose an Image

Run an Instance

Terminate the Instance

Using EBS

Creating and Deleting Volumes

Using Volumes

Using Snapshots