Getting started with OpenStack Nova

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(Floating IPs)
(Splitting Out The Compute Node)
Line 248: Line 248:
 
  mysql> select * from migrate_version;
 
  mysql> select * from migrate_version;
  
== Splitting Out The Compute Node ==
+
== Adding a Compute Node ==
  
Okay, everything so far has been done on a single node. A straightforward next step would be to use a separate machine for running VMs. It's still only a single compute node, but we'll add more compute nodes next.
+
Okay, everything so far has been done on a single node. The next step is to add another node for running VMs.
  
 
Let's assume the machine you've set up above is called 'controller' and the new machine is called 'node'.
 
Let's assume the machine you've set up above is called 'controller' and the new machine is called 'node'.
  
First, disable the network and compute services on controller:
+
First, open the rabbitmq, MySQL and Nova API ports on controller:
 
+
$ controller> for svc in compute network; do sudo service openstack-nova-$svc stop; done
+
$ controller> for svc in compute network; do sudo chkconfig openstack-nova-$svc off; done
+
 
+
Open the rabbitmq, MySQL and Nova API ports on controller:
+
  
 
  $ controller> sudo lokkit -p 3306:tcp
 
  $ controller> sudo lokkit -p 3306:tcp
Line 266: Line 261:
 
  $ controller> sudo service libvirtd reload
 
  $ controller> sudo service libvirtd reload
  
Make sure that ntp is enabled on both machines:
+
Then make sure that ntp is enabled on both machines:
  
 
  $> sudo yum install -y ntp
 
  $> sudo yum install -y ntp
Line 272: Line 267:
 
  $> sudo chkconfig ntpd on
 
  $> sudo chkconfig ntpd on
  
Install libvirt, rabbitmq and nova on node:
+
Install libvirt and nova on node:
  
 
  $ node> sudo yum install --enablerepo=updates-testing openstack-nova
 
  $ node> sudo yum install --enablerepo=updates-testing openstack-nova
$ node> sudo yum downgrade rabbitmq-server
 
$ node> sudo service rabbitmq-server start
 
$ node> sudo chkconfig rabbitmq-server on
 
 
  $ node> sudo service libvirtd start
 
  $ node> sudo service libvirtd start
 
  $ node> sudo chkconfig libvirtd on
 
  $ node> sudo chkconfig libvirtd on
Line 288: Line 280:
 
  $ node> sudo bash -c 'echo "--glance_api_servers=controller:9292" >> /etc/nova/nova.conf'
 
  $ node> sudo bash -c 'echo "--glance_api_servers=controller:9292" >> /etc/nova/nova.conf'
  
Enable the compute and network services:
+
Enable the compute service:
  
 
  $ node> for svc in compute network; do sudo service openstack-nova-$svc start; done
 
  $ node> for svc in compute network; do sudo service openstack-nova-$svc start; done
$ node> for svc in compute network; do sudo chkconfig openstack-nova-$svc on; done
 
  
Finally, make sure the network you originally created is associated with the host where the network service is now running:
+
Finally, you need to make sure the network is configured with a physical bridge interface:
  
  $ controller> sudo nova-manage network modify 10.0.0.0/24 --host=node
+
  $ controller> sudo nova-manage network create markmc 10.0.0.0/24 --bridge=br0 --bridge_interface=em1
  
Now everything should be running as before, except the VMs are launched on node.
+
Now everything should be running as before, except the VMs are launched either on controller or node.
  
 
[[Category:Cloud SIG]]
 
[[Category:Cloud SIG]]

Revision as of 07:37, 27 September 2011

Contents

Initial Installation

To get started with OpenStack's Compute Service (Nova), you can install it on Fedora 16:

$> sudo yum install --enablerepo=updates-testing openstack-nova

Nova requires the RabbitMQ AMQP messaging server to be running. Unfortunately, until bug #738067 is fixed you'll need to downgrade to 2.2.0:

$> sudo yum downgrade rabbitmq-server
$> sudo service rabbitmq-server start
$> sudo chkconfig rabbitmq-server on

Nova requires the libvirtd server to be running:

$> sudo service libvirtd start
$> sudo chkconfig libvirtd on

Unfortunately, SELinux currently needs to be disabled:

$> sudo setenforce 0

Next, you should enable the Glance API and registry services:

$> for svc in api registry; do sudo service openstack-glance-$svc start; done
$> for svc in api registry; do sudo chkconfig openstack-glance-$svc on; done

The openstack-nova-volume service requires an LVM Volume Group called nova-volumes to exist. We simply create this using a loopback sparse disk image.

$> sudo dd if=/dev/zero of=/var/lib/nova/nova-volumes.img bs=1M seek=20k count=0
$> sudo vgcreate nova-volumes $(sudo losetup --show -f /var/lib/nova/nova-volumes.img)

Now you can start the various services:

$> for svc in api objectstore compute network volume scheduler; do sudo service openstack-nova-$svc start; done
$> for svc in api objectstore compute network volume scheduler; do sudo chkconfig openstack-nova-$svc on; done

Check that all the services started up correctly and look in the logs in /var/log/nova for errors. If there are none, then Nova is up and running!

Admin User, Project and Network Setup

Now you should create an admin user, project and network. I'm going to name them all after myself:

$> sudo nova-manage user admin markmc
$> sudo nova-manage project create markmc markmc
$> sudo nova-manage network create markmc 10.0.0.0/24 1 256 --bridge=br0

Then download a set of credentials for this user/project:

$> sudo nova-manage project zipfile markmc markmc
$> sudo chmod 600 nova.zip
$> sudo chown markmc:markmc nova.zip

Unpack the credentials, source the novarc and add an SSH keypair:

$> mkdir novacreds && cd novacreds
$> unzip ../nova.zip
$> . ./novarc
$> ssh-keygen -f nova_key
$> euca-add-keypair nova_key > nova_key.priv
$> chmod 600 nova*

Images

To run an instance, you're going to need an image. Two options are described below:

  1. Building a Fedora 15 image using Oz
  2. Downloading Ubuntu based minimal images used by OpenStack developers for testing

Building an Image With Oz

You can very easily build an image using Oz. First, make sure it's installed:

$> sudo yum install /usr/bin/oz-install

Create a template definition file called f15.tdl containing:

<template>
 <name>fedora15_x86_64</name>
 <description>My Fedora 15 x86_64 template</description>
 <os>
  <name>Fedora</name>
  <version>15</version>
  <arch>x86_64</arch>
  <install type='url'>
    <url>http://download.fedoraproject.org/pub/fedora/linux/releases/15/Fedora/x86_64/os/</url>
  </install>
 </os>
 <commands>
   <command name='setup-rc-local'>
sed -i 's/rhgb quiet/console=ttyS0/' /boot/grub/grub.conf
 
cat >> /etc/rc.local &lt;&lt; EOF
if [ ! -d /root/.ssh ]; then
  mkdir -p /root/.ssh
  chmod 700 /root/.ssh
fi
 
# Fetch public key using HTTP
ATTEMPTS=10
while [ ! -f /root/.ssh/authorized_keys ]; do
    curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/aws-key 2>/dev/null
    if [ \$? -eq 0 ]; then
        cat /tmp/aws-key >> /root/.ssh/authorized_keys
        chmod 0600 /root/.ssh/authorized_keys
        restorecon /root/.ssh/authorized_keys
        rm -f /tmp/aws-key
        echo "Successfully retrieved AWS public key from instance metadata"
    else
        FAILED=\$((\$FAILED + 1))
        if [ \$FAILED -ge \$ATTEMPTS ]; then
            echo "Failed to retrieve AWS public key after \$FAILED attempts, quitting"
            break
        fi
        echo "Could not retrieve AWS public key (attempt #\$FAILED/\$ATTEMPTS), retrying in 5 seconds..."
        sleep 5
    fi
done
EOF
   </command>
 </commands>
</template>
 

Then simply do:

$> sudo oz-install -d4 -u f15.tdl

One built, you simply have to register the image with Nova:

$> sudo nova-manage image image_register /var/lib/libvirt/images/fedora15_x86_64.dsk markmc f15
$> glance index

The last command should return a list of the images registered with the Glance image registry.

Downloading Existing Images

If you don't want to build an image, just download this set of images commonly used by OpenStack developers for testing and register them with Nova:

$> mkdir images
$> curl http://images.ansolabs.com/tty.tgz | tar xvfzo -
$> cd ..
$> sudo nova-manage image convert images/

Launch an Instance

As a last step before launching, make sure the nbd kernel module is loaded so that injecting SSH key files into the filesystem on the qcow2 image works:

$> sudo modprobe nbd

You should now be able to launch an image:

$> euca-run-instances f15 -k nova_key

Or, in the case of the downloaded images:

$> euca-run-instances ami-tty --kernel aki-tty --ramdisk ari-tty -k nova_key

And then observe the instance running, observe the KVM VM running and SSH into the instance:

$> euca-describe-instances
$> sudo virsh list
$> ssh -i nova_key.priv root@10.0.0.2
$> euca-get-console-output i-00000001
$> euca-terminate-instances i-00000001

Floating IPs

You may carve out a block of public IPs and assign them to instances.

First thing you need to do is make sure that nova is configured with the correct public network interface. The default is eth0, but you can change it by e.g.

$> sudo bash -c 'echo "--public_interface=em1" >> /etc/nova/nova.conf'
$> sudo service openstack-nova-network restart

Then you can do e.g.

$> nova-manage floating create 172.31.0.224/28
$> euca-allocate-address
$> euca-associate-address -i i-00000012 172.31.0.224
$> ssh -i nova_key.priv root@172.31.0.224
$> euca-disassociate-address 172.31.0.224
$> euca-release-address 172.31.0.224

Smoke Tests

Nova comes with a selection of fairly basic smoke tests which you can run against your installation. It can be useful to use these to sanity check your configuration.

First off, you need the nova-adminclient python library which isn't yet packaged:

$> sudo pip-python install nova-adminclient

Then you need a user and project both named admin:

$> sudo nova-manage user admin admin
$> sudo nova-manage project create admin admin
$> sudo nova-manage project zipfile admin admin
$> unzip nova.zip
$> . ./novarc

Make sure you have the tty images imported as described above. You also need a block of floating IPs created, also as described above.

Then, run the tests from a fedpkg checkout:

$> fedpkg clone openstack-nova
$> cd openstack-nova
$> fedpkg switch-branch f16
$> fedpkg prep
$> cd nova-2011.3/smoketests
$> python ./run_tests.py

All the tests should pass.

Using MySQL

Rather than using the default sqlite DB, you can use MySQL.

First install and enable MySQL:

$> sudo yum install -y mysql-server
$> sudo service mysqld start
$> sudo chkconfig mysqld on

Set a password for the root account and delete the anonymous accounts:

$> mysql -u root
mysql> update mysql.user set password = password('iamroot') where user = 'root';
mysql> delete from mysql.user where user = ;

Create a database and user account specifically for nova:

mysql> create database nova;
mysql> create user 'nova'@'localhost' identified by 'nova';
mysql> create user 'nova'@'%' identified by 'nova';
mysql> grant all on nova.* to 'nova'@'%';

(If anyone can explain why nova@localhost is required even though the anonymous accounts have been deleted, I'd be very grateful :-)

Then configure nova to use the DB and install the schema:

$> sudo bash -c 'echo "--sql_connection=mysql://nova:nova@localhost/nova" >> /etc/nova/nova.conf'
$> sudo nova-manage db sync

As a final sanity check:

$> mysql -u nova -p nova
Enter password:
mysql> select * from migrate_version;

Adding a Compute Node

Okay, everything so far has been done on a single node. The next step is to add another node for running VMs.

Let's assume the machine you've set up above is called 'controller' and the new machine is called 'node'.

First, open the rabbitmq, MySQL and Nova API ports on controller:

$ controller> sudo lokkit -p 3306:tcp
$ controller> sudo lokkit -p 5672:tcp
$ controller> sudo lokkit -p 9292:tcp
$ controller> sudo service libvirtd reload

Then make sure that ntp is enabled on both machines:

$> sudo yum install -y ntp
$> sudo service ntpd start
$> sudo chkconfig ntpd on

Install libvirt and nova on node:

$ node> sudo yum install --enablerepo=updates-testing openstack-nova
$ node> sudo service libvirtd start
$ node> sudo chkconfig libvirtd on
$ node> sudo setenforce 0

Configure nova so that node can find the services on controller:

$ node> sudo bash -c 'echo "--rabbit_host=controller" >> /etc/nova/nova.conf'
$ node> sudo bash -c 'echo "--sql_connection=mysql://nova:nova@controller/nova" >> /etc/nova/nova.conf'
$ node> sudo bash -c 'echo "--glance_api_servers=controller:9292" >> /etc/nova/nova.conf'

Enable the compute service:

$ node> for svc in compute network; do sudo service openstack-nova-$svc start; done

Finally, you need to make sure the network is configured with a physical bridge interface:

$ controller> sudo nova-manage network create markmc 10.0.0.0/24 --bridge=br0 --bridge_interface=em1

Now everything should be running as before, except the VMs are launched either on controller or node.