Getting started with OpenStack EPEL

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(46 intermediate revisions by 13 users not shown)
Line 1: Line 1:
 
= Basic Setup =
 
= Basic Setup =
  
These steps will setup OpenStack nova, glance, and keystone to be accessed by the OpenStack dashboard web UI on a single host, as well as launching our first instance (virtual machine).
+
These steps will setup OpenStack services to be accessed by the OpenStack dashboard web UI on a *single node*,
 +
and also show steps to launch our first instance (virtual machine).
  
 
Many of the examples here require 'sudo' to be properly configured, please see [[Configuring Sudo]] if you need help.
 
Many of the examples here require 'sudo' to be properly configured, please see [[Configuring Sudo]] if you need help.
  
== Initial Installation ==
+
== Enable EPEL repositories ==
  
$>wget http://pbrady.fedorapeople.org/openstack-el6/epel-openstack-preview.repo
+
Follow [[EPEL#How_can_I_use_these_extra_packages.3F| EPEL instructions]], and also enable the epel-testing repository.
$>sudo mv epel-openstack-preview.repo /etc/yum.repos.d
+
  
Note that until EPEL Django gets updated to 1.3 or later, dashboard will not install correctly.
+
If on RHEL6, do not forget to enable 'optional' repository as described there!
To test,  use the RPM from
+
$> wget http://kojipkgs.fedoraproject.org/packages/Django/1.3.1/1.el6/noarch/Django-1.3.1-1.el6.noarch.rpm
+
  
First let's pull in OpenStack and some optional dependencies:
+
First let's pull in some preliminary dependencies
  
  $> sudo yum install openstack-nova openstack-glance openstack-keystone openstack-quantum \
+
  $> sudo yum install -y openstack-utils qpid-cpp-server avahi
openstack-dashboard openstack-utils qpid-cpp-server python-paste-deploy python-routes
+
 
 +
== Adjust qpidd authentication ==
  
 
Ensure auth=no is set in /etc/qpidd.conf
 
Ensure auth=no is set in /etc/qpidd.conf
  
Set selinux in permissive mode:
+
== Disable mDNS (optional) ==
>$ sudo setenforce permissive
+
  
Otherwise you will get issues like https://bugzilla.redhat.com/show_bug.cgi?id=734346
+
As per [https://bugzilla.redhat.com/show_bug.cgi?id=746111 Bug #746111], unless you have installed and are using avahi-libs, you should disable mdns_adv in /etc/libvirt/libvirtd.conf
    /usr/bin/nova-dhcpbridge: No such file or directory
+
  
if RHEL 6.2 based
+
$> sudo sed -i 's/#mdns_adv = 0/mdns_adv = 0/' /etc/libvirt/libvirtd.conf
  $> sudo openstack-config --set /etc/nova/nova.conf DEFAULT force_dhcp_release False
+
else if RHEL 6.3 based
+
  $>  sudo yum install dnsdmasq-utils
+
  
 +
(Note: This has been fixed upstream, but may still affect older versions)
  
 +
== Setup DHCP control depending on distro ==
  
Run the helper script to get MySQL configured for use with openstack-nova. If <code>mysql-server</code> is not already installed, this script will install it for you.
+
On RHEL 6.2 or below:
 +
  $> sudo openstack-config --set /etc/nova/nova.conf DEFAULT force_dhcp_release False
 +
On RHEL 6.3 or above:
 +
  $> sudo yum install dnsmasq-utils  # from the Red Hat '''optional''' channel
  
$> sudo openstack-nova-db-setup
+
== Start support services ==
  
Similarly, run the helper script to get MySQL configured for use with openstack-glance.
+
Nova requires dbus to be running:
  
  $> sudo openstack-glance-db-setup
+
  $> sudo service messagebus start && sudo chkconfig messagebus on
  
Nova requires the QPID messaging server to be running.
+
== Install and Setup OpenStack on a single node ==
  
$> sudo service qpidd start && sudo chkconfig qpidd on
+
This script will install nova, glance, quantum, cinder, swift, keystone, dashboard.
 +
Note it configures and enables all but the quantum service at this time
  
Nova requires the libvirtd server to be running:
+
  sudo [https://github.com/fedora-openstack/openstack-utils/blob/master/utils/openstack-demo-install openstack-demo-install]
  
$> sudo service libvirtd start && sudo chkconfig libvirtd on
+
Now copy the keystonerc file created by the install script above to your home directory, so that it can be easily sourced to provide the credentials used by the various OpenStack command line utilities.
  
Next, you should enable the Glance API and registry services:
+
  sudo cp /root/keystonerc ~
$> for svc in api registry; do sudo service openstack-glance-$svc start; sudo chkconfig openstack-glance-$svc on  ; done
+
  
 +
== Installing within a VM ==
  
The openstack-nova-volume service requires an LVM Volume Group called nova-volumes to exist. We simply create this using a loopback sparse disk image.
+
If you are testing OpenStack in a virtual machine, openstack-demo-install will have
 
+
already detected this and configured nova to use qemu without KVM and hardware virtualization.
$> sudo dd if=/dev/zero of=/var/lib/nova/nova-volumes.img bs=1M seek=20k count=0
+
$> sudo vgcreate nova-volumes $(sudo losetup --show -f /var/lib/nova/nova-volumes.img)
+
 
+
If you are testing OpenStack in a virtual machine, you need to configure nova to use qemu without KVM and hardware virtualization.
+
The last 2 commands here work around a libvirt issue fixed in RHEL 6.4.
+
 
Note nested virtualization will be the much slower TCG variety, and you should provide lots of memory to the top level guest,
 
Note nested virtualization will be the much slower TCG variety, and you should provide lots of memory to the top level guest,
 
as the openstack created guests default to 2GM RAM with no overcommit.
 
as the openstack created guests default to 2GM RAM with no overcommit.
  
$> sudo openstack-config-set /etc/nova/nova.conf DEFAULT libvirt_type qemu
+
We still need some manual config to work around a libvirt issue fixed in RHEL 6.4.
 +
 
 
  $> sudo ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-system-x86_64
 
  $> sudo ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-system-x86_64
  $> service libvirtd restart
+
  $> sudo service libvirtd restart
 +
$> sudo service openstack-nova-compute restart
  
Now you can start the various services:
+
== Enable external access to OpenStack Dashboard ==
  
  $> for svc in api objectstore compute network volume scheduler cert; do sudo service openstack-nova-$svc start ; sudo chkconfig openstack-nova-$svc on ;  done
+
To open up the firewall ports for HTTP:
 +
  $> sudo lokkit -p http:tcp
 +
$> sudo lokkit -p https:tcp
  
 +
The dashboard should be accessible at http://$HOST/dashboard . Account and password should be
 +
what you configured for the keystone setup, which is displayed when openstack-demo-install completes.
  
Check that all the services started up correctly and look in the logs in <code>/var/log/nova</code> for errors. If there are none, then Nova is up and running!
+
= Basic Folsom Operations =
  
== Initial Keystone setup ==
+
The operational instructions below have not been updated for Folsom at present.
 +
For Folsom specific details please see [https://fedoraproject.org/wiki/Test_Day:2012-09-18_OpenStack#Setup_OpenStack_volumes OpenStack Folsom setup and recipes]
  
Keystone is the openstack identity service, providing a central place to
+
= Basic OpenStack Operations =
set up openstack users, groups, and accounts that can be shared across all
+
other services. This deprecates the old style user accounts manually set
+
up with nova-manage.
+
  
Setting up keystone is required for using the Openstack dashboard.
+
The items below have only been tested on the Essex version of OpenStack,
 +
and may need adjustment for OpenStack Folsom.
  
* Configure the Keystone database, similar to how we do it for nova
+
== Setup volume storage ==
$> sudo openstack-keystone-db-setup
+
  
* Set up a keystonerc file with a generated admin token and various passwords:
+
The openstack-nova-volume service requires an LVM Volume Group called nova-volumes to exist. We simply create this using a loopback sparse disk image.
$> cat > keystonerc <<EOF
+
export ADMIN_TOKEN=$(openssl rand -hex 10)
+
export OS_USERNAME=admin
+
export OS_PASSWORD=verybadpass
+
export OS_TENANT_NAME=admin
+
export OS_AUTH_URL=http://127.0.0.1:5000/v2.0/
+
EOF
+
$> . ./keystonerc
+
  
* Set the administrative token in the config file
+
$> sudo dd if=/dev/zero of=/var/lib/nova/nova-volumes.img bs=1M seek=20k count=0
  $> sudo openstack-config-set /etc/keystone/keystone.conf DEFAULT admin_token $ADMIN_TOKEN
+
  $> sudo vgcreate nova-volumes $(sudo losetup --show -f /var/lib/nova/nova-volumes.img)
  
* Start and enable Keystone service
 
$> sudo service openstack-keystone start && sudo chkconfig openstack-keystone on
 
 
* Create sample Tenants, Users and Roles
 
$> sudo ADMIN_PASSWORD=$OS_PASSWORD SERVICE_PASSWORD=servicepass openstack-keystone-sample-data
 
 
* Test the Keystone CLI is working
 
$> keystone user-list
 
+----------------------------------+---------+-------------------+-------+
 
|                id                | enabled |      email      |  name |
 
+----------------------------------+---------+-------------------+-------+
 
| 05742d10109540d2892d17ec312a6cd9 | True    | admin@example.com | admin |
 
| 25fe47659d6a4255a663e6add1979d6c | True    | admin@example.com | demo  |
 
+----------------------------------+---------+-------------------+-------+
 
 
== Configure nova to use keystone ==
 
 
* Change nova configuration to use keystone:
 
$> sudo openstack-config-set /etc/nova/api-paste.ini filter:authtoken admin_tenant_name service
 
$> sudo openstack-config-set /etc/nova/api-paste.ini filter:authtoken admin_user nova
 
$> sudo openstack-config-set /etc/nova/api-paste.ini filter:authtoken admin_password servicepass
 
$> sudo openstack-config-set /etc/nova/nova.conf DEFAULT auth_strategy keystone
 
$> sudo service openstack-nova-api restart
 
 
* Verify that nova can talk with keystone (requires OS_* exports from previous keystone section)
 
 
$> nova flavor-list
 
+----+-----------+-----------+------+----------+-------+-------------+
 
| ID |    Name  | Memory_MB | Swap | Local_GB | VCPUs | RXTX_Factor |
 
+----+-----------+-----------+------+----------+-------+-------------+
 
| 1  | m1.tiny  | 512      |      | 0        | 1    | 1.0        |
 
| 2  | m1.small  | 2048      |      | 10      | 1    | 1.0        |
 
| 3  | m1.medium | 4096      |      | 10      | 2    | 1.0        |
 
| 4  | m1.large  | 8192      |      | 10      | 4    | 1.0        |
 
| 5  | m1.xlarge | 16384    |      | 10      | 8    | 1.0        |
 
+----+-----------+-----------+------+----------+-------+-------------+
 
 
== Configure glance to use keystone ==
 
 
* Change glance configuration to use keystone:
 
$> sudo openstack-config-set /etc/glance/glance-api.conf paste_deploy flavor keystone
 
$> sudo openstack-config-set /etc/glance/glance-registry.conf paste_deploy flavor keystone
 
$> sudo openstack-config-set /etc/glance/glance-api-paste.ini filter:authtoken admin_tenant_name service
 
$> sudo openstack-config-set /etc/glance/glance-api-paste.ini filter:authtoken admin_user glance
 
$> sudo openstack-config-set /etc/glance/glance-api-paste.ini filter:authtoken admin_password servicepass
 
$> sudo openstack-config-set /etc/glance/glance-registry-paste.ini filter:authtoken admin_tenant_name service
 
$> sudo openstack-config-set /etc/glance/glance-registry-paste.ini filter:authtoken admin_user glance
 
$> sudo openstack-config-set /etc/glance/glance-registry-paste.ini filter:authtoken admin_password servicepass
 
$> sudo service openstack-glance-api restart
 
$> sudo service openstack-glance-registry restart
 
 
* Verify that glance can talk with keystone (requires OS_* exports from the previous keystone section)
 
 
$> glance index
 
  
 
== Nova Network Setup ==
 
== Nova Network Setup ==
Line 165: Line 102:
 
== Register an Image ==
 
== Register an Image ==
  
To run an instance, you are going to need an image. There are prebuilt Fedora 16 JEOS (Just Enough OS) images that can be downloaded.  
+
To run an instance, you are going to need an image. There are prebuilt Fedora 17 JEOS (Just Enough OS) images that can be downloaded.  
Note this will download a 200MB image (without a progress bar)
+
Note this will download a 250MB image (without a progress bar)
  
   $> glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=ovf \
+
   $> glance add name=f17-jeos is_public=true disk_format=qcow2 container_format=ovf \
       copy_from=http://berrange.fedorapeople.org/images/2012-02-29/f16-x86_64-openstack-sda.qcow2
+
       copy_from=http://berrange.fedorapeople.org/images/2012-11-15/f17-x86_64-openstack-sda.qcow2
  
 
== Launch an Instance ==
 
== Launch an Instance ==
Line 178: Line 115:
 
  $> chmod 600 oskey.priv
 
  $> chmod 600 oskey.priv
  
Modify key injection code:
+
Configure key injection mode, to allow guestfs to inject into multiple guest types:
  There is currently a mismatch between how various distributions expose volumes for mounting.  The code that calls ''guestfs''  needs a tweak so that it works corectly for multiple Guest types. Edit the file
+
  $> sudo openstack-config --set /etc/nova/nova.conf DEFAULT libvirt_inject_partition -1
/usr/lib/python2.6/site-packages/nova/virt/disk/guestfs.py, line 37, applying the  following change:
+
$> sudo service openstack-nova-compute restart
      def mnt_dev(self):
+
          try:
+
  -           partition = int(self.partition or 0)
+
  +            partition = -1
+
          except ValueError:
+
              self.error = _('unsupported partition: %s') % self.partition
+
              return False
+
 
+
  
 
Launch an instance:
 
Launch an instance:
  
 
  $> nova boot myserver --flavor 2 --key_name mykey \
 
  $> nova boot myserver --flavor 2 --key_name mykey \
       --image $(glance index | grep f16-jeos | awk '{print $1}')
+
       --image $(glance index | grep f17-jeos | awk '{print $1}')
  
 
And then observe the instance running, observe the KVM VM running and SSH into the instance:
 
And then observe the instance running, observe the KVM VM running and SSH into the instance:
Line 199: Line 128:
 
  $> sudo virsh list
 
  $> sudo virsh list
 
  $> nova list
 
  $> nova list
  $> ssh -i oskey.priv ec2-user@10.0.0.2
+
  $> ssh -i oskey.priv root@10.0.0.2 # use ec2-user instead of root if launching the image f17-jeos
 
  $> nova console-log myserver
 
  $> nova console-log myserver
 
  $> nova delete myserver
 
  $> nova delete myserver
 
== Configure the OpenStack Dashboard ==
 
 
The OpenStack dashboard is the official web user interface for OpenStack. It should mostly work out of the box, as long as keystone has been configured properly.
 
 
* Install the dashboard
 
$> sudo yum install openstack-dashboard
 
 
* Make sure httpd is running
 
$> sudo service httpd restart
 
$> sudo chkconfig httpd on
 
 
* If selinux is enabled, you will have to allow httpd to access other network services (the dashboard talks to the http API of the other OpenStack services)
 
$> sudo setsebool -P httpd_can_network_connect=on
 
 
The dashboard should then be accessed with a web browser at http://localhost/dashboard . Account and password should be
 
what you configured for the keystone setup.
 
 
 
 
To open up the firewall ports for HTTP, edit /etc/sysconfig/iptables and add the following two lines:
 
 
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
 
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
 
 
just before the line:
 
 
-A INPUT -j REJECT --reject-with icmp-host-prohibited
 
 
And restart iptables:
 
>$ sudo service iptables restart
 
 
 
== Configure swift with keystone ==
 
These are the minimal steps required to setup a swift installation on RHEL which keystone authentication, this wouldn't be considered a working swift system but at the very least will provide you with a working swift API to test clients against, most notibly it doesn't include replication, multiple zones and loadbalancing
 
 
Installing swift from epel-testing for now (the older one ignores the OS_* env variables)
 
$> sudo yum install --enablerepo=epel-testing openstack-swift openstack-swift-proxy openstack-swift-account openstack-swift-container openstack-swift-object memcached
 
 
Ensure the keystone env variables are still setup from the previous steps
 
 
We need to create 5 configuration files
 
 
$> cat > /tmp/swift.conf <<- EOF
 
[swift-hash]
 
swift_hash_path_suffix = randomestringchangeme
 
EOF
 
$> sudo mv /tmp/swift.conf /etc/swift/swift.conf
 
 
$> cat > /tmp/proxy-server.conf <<- EOF
 
[DEFAULT]
 
bind_port = 8080
 
workers = 8
 
user = swift
 
[pipeline:main]
 
pipeline = catch_errors healthcheck cache authtoken keystone proxy-server
 
[app:proxy-server]
 
use = egg:swift#proxy
 
account_autocreate = true
 
[filter:keystone]
 
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
 
operator_roles = admin, swiftoperator
 
[filter:authtoken]
 
paste.filter_factory = keystone.middleware.auth_token:filter_factory
 
auth_port = 35357
 
auth_host = 127.0.0.1
 
auth_protocol = http
 
admin_token = ADMINTOKEN
 
#  ??? Are these needed?
 
service_port = 5000
 
service_host = 127.0.0.1
 
service_protocol = http
 
auth_token = ADMINTOKEN
 
[filter:healthcheck]
 
use = egg:swift#healthcheck
 
[filter:cache]
 
use = egg:swift#memcache
 
memcache_servers = 127.0.0.1:11211
 
[filter:catch_errors]
 
use = egg:swift#catch_errors
 
EOF
 
$> sudo mv /tmp/proxy-server.conf /etc/swift/proxy-server.conf
 
 
$> cat > /tmp/account-server.conf <<- EOF
 
[DEFAULT]
 
bind_ip = 127.0.0.1
 
workers = 2
 
[pipeline:main]
 
pipeline = account-server
 
[app:account-server]
 
use = egg:swift#account
 
[account-replicator]
 
[account-auditor]
 
[account-reaper]
 
EOF
 
$> sudo mv /tmp/account-server.conf /etc/swift/account-server.conf
 
 
$> cat > /tmp/container-server.conf <<- EOF
 
[DEFAULT]
 
bind_ip = 127.0.0.1
 
workers = 2
 
[pipeline:main]
 
pipeline = container-server
 
[app:container-server]
 
use = egg:swift#container
 
[container-replicator]
 
[container-updater]
 
[container-auditor]
 
EOF
 
$> sudo mv /tmp/container-server.conf /etc/swift/container-server.conf
 
 
$> cat > /tmp/object-server.conf <<- EOF
 
[DEFAULT]
 
bind_ip = 127.0.0.1
 
workers = 2
 
[pipeline:main]
 
pipeline = object-server
 
[app:object-server]
 
use = egg:swift#object
 
[object-replicator]
 
[object-updater]
 
[object-auditor]
 
EOF
 
$> sudo mv /tmp/object-server.conf /etc/swift/object-server.conf
 
 
So that swift can authenticate tokens we need to set the keystone Admin token in the swift proxy file
 
$> sudo openstack-config-set /etc/swift/proxy-server.conf filter:authtoken admin_token $ADMIN_TOKEN
 
$> sudo openstack-config-set /etc/swift/proxy-server.conf filter:authtoken auth_token $ADMIN_TOKEN
 
 
Create the stoage device for swift, these instructions use a loopback device but a physical devive or logical volume can be used
 
$> truncate --size=20G /tmp/swiftstorage
 
$> DEVICE=$(sudo losetup --show -f /tmp/swiftstorage)
 
$> sudo mkfs.ext4 -I 1024 $DEVICE
 
$> sudo mkdir -p /srv/node/partitions
 
$> sudo mount $DEVICE /srv/node/partitions -t ext4 -o noatime,nodiratime,nobarrier,user_xattr
 
 
$> cd /etc/swift
 
 
Create the ring, with 1024 partitions (only suitable for a small test environment) and 1 zone
 
$> sudo swift-ring-builder account.builder create 10 1 1
 
$> sudo swift-ring-builder container.builder create 10 1 1
 
$> sudo swift-ring-builder object.builder create 10 1 1
 
 
Create a device for each of the account, container and object services
 
$> sudo swift-ring-builder account.builder add z1-127.0.0.1:6002/partitions 100
 
$> sudo swift-ring-builder container.builder add z1-127.0.0.1:6001/partitions 100
 
$> sudo swift-ring-builder object.builder add z1-127.0.0.1:6000/partitions 100
 
 
Rebalance the ring (allocates partitions to devices)
 
$> sudo swift-ring-builder account.builder rebalance
 
$> sudo swift-ring-builder container.builder rebalance
 
$> sudo swift-ring-builder object.builder rebalance
 
 
make sure swift owns appropriate files
 
$> sudo chown -R swift:swift /etc/swift /srv/node/partitions
 
 
Added the swift service and endpoint to keystone
 
$> SERVICEID=$(keystone  service-create --name=swift --type=object-store --description="Swift Service" | grep "id " | cut -d "|" -f 3)
 
$> echo $SERVICEID # just making sure we got a SERVICEID
 
$> keystone endpoint-create --service_id $SERVICEID --publicurl "http://127.0.0.1:8080/v1/AUTH_\$(tenant_id)s" --adminurl "http://127.0.0.1:8080/v1/AUTH_\$(tenant_id)s" --internalurl "http://127.0.0.1:8080/v1/AUTH_\$(tenant_id)s"
 
 
Start the services
 
$> sudo /etc/init.d/memcached start
 
$> for srv in account container object proxy  ; do sudo /etc/init.d/openstack-swift-$srv start ; done
 
 
Test the swift client and upload files
 
$> swift list
 
$> swift upload container /path/to/file
 
  
 
= Additional Functionality =
 
= Additional Functionality =
Line 379: Line 140:
 
  $> . ./keystonerc
 
  $> . ./keystonerc
 
  $> USER_ID=$(keystone user-list | awk '/admin / {print $2}')
 
  $> USER_ID=$(keystone user-list | awk '/admin / {print $2}')
  $> ACCESS_KEY=$(keystone ec2-credentials-list --user $USER_ID | awk '/admin / {print $4}')
+
  $> ACCESS_KEY=$(keystone ec2-credentials-list --user-id $USER_ID | awk '/admin / {print $4}')
  $> SECRET_KEY=$(keystone ec2-credentials-list --user $USER_ID | awk '/admin / {print $6}')
+
  $> SECRET_KEY=$(keystone ec2-credentials-list --user-id $USER_ID | awk '/admin / {print $6}')
 
  $> cat > novarc <<EOF
 
  $> cat > novarc <<EOF
 
  export EC2_URL=http://localhost:8773/services/Cloud
 
  export EC2_URL=http://localhost:8773/services/Cloud
Line 391: Line 152:
 
You should now be able to launch an image:
 
You should now be able to launch an image:
  
  $> euca-run-instances f16-jeos -k nova_key
+
  $> euca-run-instances f17-jeos -k nova_key
 
  $> euca-describe-instances
 
  $> euca-describe-instances
 
  $> euca-get-console-output i-00000001
 
  $> euca-get-console-output i-00000001
Line 476: Line 237:
 
  $> mkdir images
 
  $> mkdir images
 
  $> cd images
 
  $> cd images
  $> curl http://images.ansolabs.com/tty.tgz | tar xvfzo -
+
  $> curl -L http://github.com/downloads/citrix-openstack/warehouse/tty.tgz | tar xvfzo -
 
  $> glance add name=aki-tty disk_format=aki container_format=aki is_public=true < aki-tty/image
 
  $> glance add name=aki-tty disk_format=aki container_format=aki is_public=true < aki-tty/image
 
  $> glance add name=ami-tty disk_format=ami container_format=ami is_public=true < ami-tty/image
 
  $> glance add name=ami-tty disk_format=ami container_format=ami is_public=true < ami-tty/image
Line 488: Line 249:
  
 
If you use the Chrome browser, kill it before embarking on this section, as it has been [https://bugzilla.redhat.com/show_bug.cgi?id=727925 known] to cause the lvcreate command to fail with 'incorrect semaphore state' errors.
 
If you use the Chrome browser, kill it before embarking on this section, as it has been [https://bugzilla.redhat.com/show_bug.cgi?id=727925 known] to cause the lvcreate command to fail with 'incorrect semaphore state' errors.
 +
 +
Note when setting up volumes in production, make sure you don't put your volume nodes on the same network as your guests
 +
when using the default volume driver, as all the iscsi targets are discoverable and accessible without any security.
  
 
Start the SCSI target daemon
 
Start the SCSI target daemon
Line 504: Line 268:
 
Re-run the previously terminated instance if necessary:
 
Re-run the previously terminated instance if necessary:
  
  $> INSTANCE=$(euca-run-instances f16-jeos -k mykey | grep INSTANCE | awk '{print $2}')
+
  $> INSTANCE=$(euca-run-instances f17-jeos -k mykey | grep INSTANCE | awk '{print $2}')
  
 
or:
 
or:
Line 566: Line 330:
 
  $> euca-disassociate-address 172.31.0.224
 
  $> euca-disassociate-address 172.31.0.224
 
  $> euca-release-address 172.31.0.224
 
  $> euca-release-address 172.31.0.224
 +
 +
== VNC access ==
 +
 +
To setup VNC access to guests through the dashboard:
 +
 +
nova-novncproxy reads some parameters in /etc/nova/nova.conf file.
 +
First you need to configure your cloud controller to enable VNC
 +
 +
novncproxy_host = 0.0.0.0
 +
novncproxy_port = 6080
 +
 +
and in the nova compute nodes you need something like this
 +
 +
<pre>novncproxy_base_url=http://NOVNCPROXY_FQDN:6080/vnc_auto.html
 +
vnc_enabled=true
 +
vncserver_listen=COMPUTE_FQDN
 +
vncserver_proxyclient_address=COMPUTE_FQDN</pre>
 +
 +
You should also make sure that openstack-nova-consoleauth has been started on the controller node:
 +
<pre>
 +
$ controller> sudo /etc/init.d/openstack-nova-consoleauth restart</pre>
 +
 +
After restarting nova services on both nodes the newly created machines will run the qemu-kvm with a parameter -vnc compute_fqdn:display_number.
 +
Then after starting the novncproxy and connecting to the dashboard it will discover the host and point to the novncproxy with the appropriate values and connect to the VM.
 +
 +
Note ensure than the iptables entries for VNC ports (5900+DISPLAYNUMBER) are allowed.
 +
 +
== Migrate and Resize ==
 +
 +
This is implemented currently by transferring the images between compute nodes over ssh.
 +
Therefore currently you need to make these adjustments on each compute node to allow that.
 +
 +
* Allow logins for the nova user
 +
  # usermod -s /bin/bash nova
 +
  # su - nova
 +
  $ chcon -u system_u -r object_r -t user_home_t .
 +
  $ mkdir -p -m 700 --context=system_u:object_r:ssh_home_t:s0 .ssh && cd .ssh
 +
 +
* Disable host identity checking by adding this to ssh config
 +
  $ cat > config <<EOF
 +
  Host *
 +
    StrictHostKeyChecking no
 +
    UserKnownHostsFile=/dev/null
 +
  EOF
 +
 +
* Generate and distribute ssh key
 +
  $ ssh-keygen -f id_rsa -b 1024 -P ""
 +
  $ scp /var/lib/nova/.ssh/id_rsa.pub root@otherHost:/var/lib/nova/.ssh/authorized_keys
 +
  # chown nova:nova /var/lib/nova/.ssh/authorized_keys
 +
 +
To improve the SELinux config in future the above context manipulations
 +
will need to be done centrally, so that restorecon works as expected.
 +
 +
== Live Migration of VM instances ==
 +
 +
First note the [http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html official OpenStack docs on the feature]
 +
and a [https://review.openstack.org/#/c/11172/ doc patch distinguishing libvirt live migration].
 +
 +
* Seting NFS server
 +
** Make an nfs share with no_root_squash (nova uses root-wrap to chown the instance's disk to qemu:qemu)
 +
** Make nova user and qemu user:
 +
nova:x:162:162::/home/nova:/bin/bash
 +
qemu:x:107:107::/home/qemu:/bin/bash
 +
** chown -R nova:nova /the/nfs/share
 +
 +
* Mount nfs share on each host at /var/lib/nova/instances
 +
 +
* Configure libvirt
 +
** See the [http://libvirt.org/remote.html#Remote_certificates libvirt wiki] as to how to create certificates.
 +
** Edit /etc/libvirt/libvirt.conf
 +
listen_tcp = 1
 +
tcp_port = "16509"
 +
auth_tcp = "none"
 +
** Edit /etc/sysconfig/libvirtd
 +
LIBVIRTD_ARGS="--listen"
 +
 +
* Restart libvirtd & OpenStack compute services
  
 
= Deployment =
 
= Deployment =
Line 575: Line 416:
 
Let's assume the machine you've set up above is called 'controller' and the new machine is called 'node'.
 
Let's assume the machine you've set up above is called 'controller' and the new machine is called 'node'.
  
First, open the rabbitmq, MySQL, Nova API and iSCSI ports on controller:
+
First, open the qpid, MySQL, Nova API and iSCSI ports on controller:
  
 
  $ controller> sudo lokkit -p 3306:tcp
 
  $ controller> sudo lokkit -p 3306:tcp
Line 591: Line 432:
 
Install libvirt and nova on node:
 
Install libvirt and nova on node:
  
  $ node> sudo yum install --enablerepo=updates-testing openstack-nova
+
  $ node> sudo yum install --enablerepo=epel-testing openstack-nova python-keystone openstack-utils
 
  $ node> sudo service libvirtd start
 
  $ node> sudo service libvirtd start
 
  $ node> sudo chkconfig libvirtd on
 
  $ node> sudo chkconfig libvirtd on
Line 598: Line 439:
 
Configure nova so that node can find the services on controller:
 
Configure nova so that node can find the services on controller:
  
  $ node> sudo openstack-config-set /etc/nova/nova.conf DEFAULT rabbit_host controller
+
  $ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT qpid_hostname controller
  $ node> sudo openstack-config-set /etc/nova/nova.conf DEFAULT sql_connection mysql://nova:nova@controller/nova
+
  $ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT sql_connection mysql://nova:nova@controller/nova
  $ node> sudo openstack-config-set /etc/nova/nova.conf DEFAULT glance_api_servers controller:9292
+
  $ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT glance_api_servers controller:9292
  $ node> sudo openstack-config-set /etc/nova/nova.conf DEFAULT iscsi_ip_prefix 172.31.0.107
+
  $ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT iscsi_ip_prefix 172.31.0.107
 
+
$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT auth_strategy keystone
 
(The {{{iscsi_ip_prefix}}} value is the IP address of the controller node)
 
(The {{{iscsi_ip_prefix}}} value is the IP address of the controller node)
  
Enable the compute service:
+
Configure the Network interfaces
 +
The bridge name should match what use used in the nova-manage command on the controller
 +
$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_network_bridge demonetbr0
  
  $ node> for svc in compute network; do sudo service openstack-nova-$svc start; done
+
The device which should be moved onto the bridge (nova will set up this bridge, once it done you can view it with the brctl command
 +
  $ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_interface eth0
 +
$ controller> sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_interface eth0
  
Finally, you need to make sure the network is configured with a physical bridge interface:
+
$ node> brctl show
 +
 
 +
Enable the compute service:
  
  $ controller> sudo nova-manage network create demonet 10.0.0.0/24 --bridge=demonetbr0 --bridge_interface=em1
+
  $ node> sudo service openstack-nova-compute start
  
Now everything should be running as before, except the VMs are launched either on controller or node.
+
Now everything should be running as before, except the VMs are launched either on controller or node. You will only be able to ping/ssh to vm's from the controller node.
  
 
== Manual Setup of MySQL ==
 
== Manual Setup of MySQL ==
Line 676: Line 523:
 
  $> fedpkg clone openstack-nova
 
  $> fedpkg clone openstack-nova
 
  $> cd openstack-nova
 
  $> cd openstack-nova
  $> fedpkg switch-branch f16
+
  $> fedpkg switch-branch f18
 
  $> fedpkg prep
 
  $> fedpkg prep
  $> cd nova-2011.3/smoketests
+
  $> cd nova-2012.2/smoketests
 
  $> python ./run_tests.py
 
  $> python ./run_tests.py
  
Line 748: Line 595:
 
  $> sudo service libvirtd restart
 
  $> sudo service libvirtd restart
  
[[Category:Cloud SIG]]
+
[[Category:OpenStack]]

Revision as of 12:44, 16 November 2012

Contents

Basic Setup

These steps will setup OpenStack services to be accessed by the OpenStack dashboard web UI on a *single node*, and also show steps to launch our first instance (virtual machine).

Many of the examples here require 'sudo' to be properly configured, please see Configuring Sudo if you need help.

Enable EPEL repositories

Follow EPEL instructions, and also enable the epel-testing repository.

If on RHEL6, do not forget to enable 'optional' repository as described there!

First let's pull in some preliminary dependencies

$> sudo yum install -y openstack-utils qpid-cpp-server avahi

Adjust qpidd authentication

Ensure auth=no is set in /etc/qpidd.conf

Disable mDNS (optional)

As per Bug #746111, unless you have installed and are using avahi-libs, you should disable mdns_adv in /etc/libvirt/libvirtd.conf

$> sudo sed -i 's/#mdns_adv = 0/mdns_adv = 0/' /etc/libvirt/libvirtd.conf

(Note: This has been fixed upstream, but may still affect older versions)

Setup DHCP control depending on distro

On RHEL 6.2 or below:

 $> sudo openstack-config --set /etc/nova/nova.conf DEFAULT force_dhcp_release False

On RHEL 6.3 or above:

 $> sudo yum install dnsmasq-utils  # from the Red Hat optional channel

Start support services

Nova requires dbus to be running:

$> sudo service messagebus start && sudo chkconfig messagebus on

Install and Setup OpenStack on a single node

This script will install nova, glance, quantum, cinder, swift, keystone, dashboard. Note it configures and enables all but the quantum service at this time

 sudo openstack-demo-install

Now copy the keystonerc file created by the install script above to your home directory, so that it can be easily sourced to provide the credentials used by the various OpenStack command line utilities.

 sudo cp /root/keystonerc ~

Installing within a VM

If you are testing OpenStack in a virtual machine, openstack-demo-install will have already detected this and configured nova to use qemu without KVM and hardware virtualization. Note nested virtualization will be the much slower TCG variety, and you should provide lots of memory to the top level guest, as the openstack created guests default to 2GM RAM with no overcommit.

We still need some manual config to work around a libvirt issue fixed in RHEL 6.4.

$> sudo ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-system-x86_64
$> sudo service libvirtd restart
$> sudo service openstack-nova-compute restart

Enable external access to OpenStack Dashboard

To open up the firewall ports for HTTP:

$> sudo lokkit -p http:tcp
$> sudo lokkit -p https:tcp

The dashboard should be accessible at http://$HOST/dashboard . Account and password should be what you configured for the keystone setup, which is displayed when openstack-demo-install completes.

Basic Folsom Operations

The operational instructions below have not been updated for Folsom at present. For Folsom specific details please see OpenStack Folsom setup and recipes

Basic OpenStack Operations

The items below have only been tested on the Essex version of OpenStack, and may need adjustment for OpenStack Folsom.

Setup volume storage

The openstack-nova-volume service requires an LVM Volume Group called nova-volumes to exist. We simply create this using a loopback sparse disk image.

$> sudo dd if=/dev/zero of=/var/lib/nova/nova-volumes.img bs=1M seek=20k count=0
$> sudo vgcreate nova-volumes $(sudo losetup --show -f /var/lib/nova/nova-volumes.img)


Nova Network Setup

To create the network do:

$> sudo nova-manage network create demonet 10.0.0.0/24 1 256 --bridge=demonetbr0

NB the network range here, should *not* be the one used on your existing physical network. It should be a range dedicated for the network that OpenStack will configure. So if 10.0.0.0/24 clashes with your local network, pick another range

Register an Image

To run an instance, you are going to need an image. There are prebuilt Fedora 17 JEOS (Just Enough OS) images that can be downloaded. Note this will download a 250MB image (without a progress bar)

 $> glance add name=f17-jeos is_public=true disk_format=qcow2 container_format=ovf \
      copy_from=http://berrange.fedorapeople.org/images/2012-11-15/f17-x86_64-openstack-sda.qcow2

Launch an Instance

Create a keypair:

$> nova keypair-add mykey > oskey.priv
$> chmod 600 oskey.priv

Configure key injection mode, to allow guestfs to inject into multiple guest types:

$> sudo openstack-config --set /etc/nova/nova.conf DEFAULT libvirt_inject_partition -1
$> sudo service openstack-nova-compute restart

Launch an instance:

$> nova boot myserver --flavor 2 --key_name mykey \
     --image $(glance index | grep f17-jeos | awk '{print $1}')

And then observe the instance running, observe the KVM VM running and SSH into the instance:

$> sudo virsh list
$> nova list
$> ssh -i oskey.priv root@10.0.0.2  # use ec2-user instead of root if launching the image f17-jeos
$> nova console-log myserver
$> nova delete myserver

Additional Functionality

Using Eucalyptus tools

Set up a rc file for EC2 access (this expects a prior keystone configuration)

$> . ./keystonerc
$> USER_ID=$(keystone user-list | awk '/admin / {print $2}')
$> ACCESS_KEY=$(keystone ec2-credentials-list --user-id $USER_ID | awk '/admin / {print $4}')
$> SECRET_KEY=$(keystone ec2-credentials-list --user-id $USER_ID | awk '/admin / {print $6}')
$> cat > novarc <<EOF
export EC2_URL=http://localhost:8773/services/Cloud
export EC2_ACCESS_KEY=$ACCESS_KEY
export EC2_SECRET_KEY=$SECRET_KEY
EOF
$> chmod 600 novarc
$> . ./novarc 

You should now be able to launch an image:

$> euca-run-instances f17-jeos -k nova_key
$> euca-describe-instances
$> euca-get-console-output i-00000001
$> euca-terminate-instances i-00000001

Images

Rather than the prebuilt Fedora 16 JEOS image referenced above, there are other image options.

  1. Building a Fedora 16 JEOS image using Oz
  2. Downloading ttylinux based minimal images used by OpenStack developers for testing

Building Fedora 16 JEOS Images With Oz

You can very easily build an image using Oz. First, make sure it's installed:

$> sudo yum install /usr/bin/oz-install

Create a template definition file called f16-jeos.tdl containing:

<template>
 <name>fedora16_x86_64</name>
 <description>My Fedora 16 x86_64 template</description>
 <os>
  <name>Fedora</name>
  <version>16</version>
  <arch>x86_64</arch>
  <install type='url'>
    <url>http://download.fedoraproject.org/pub/fedora/linux/releases/16/Fedora/x86_64/os/</url>
  </install>
 </os>
 <commands>
   <command name='setup-rc-local'>
sed -i 's/rhgb quiet/console=ttyS0/' /boot/grub/grub.conf
 
cat >> /etc/rc.local &lt;&lt; EOF
if [ ! -d /root/.ssh ]; then
  mkdir -p /root/.ssh
  chmod 700 /root/.ssh
fi
 
# Fetch public key using HTTP
ATTEMPTS=10
while [ ! -f /root/.ssh/authorized_keys ]; do
    curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/aws-key 2>/dev/null
    if [ \$? -eq 0 ]; then
        cat /tmp/aws-key >> /root/.ssh/authorized_keys
        chmod 0600 /root/.ssh/authorized_keys
        restorecon /root/.ssh/authorized_keys
        rm -f /tmp/aws-key
        echo "Successfully retrieved AWS public key from instance metadata"
    else
        FAILED=\$((\$FAILED + 1))
        if [ \$FAILED -ge \$ATTEMPTS ]; then
            echo "Failed to retrieve AWS public key after \$FAILED attempts, quitting"
            break
        fi
        echo "Could not retrieve AWS public key (attempt #\$FAILED/\$ATTEMPTS), retrying in 5 seconds..."
        sleep 5
    fi
done
EOF
   </command>
 </commands>
</template>
 

Then simply do:

$> sudo oz-install -d4 -u f16-jeos.tdl

Once built, you simply have to register the image with Nova:

$> glance add name=f16-jeos is_public=true container_format=bare disk_format=raw < /var/lib/libvirt/images/fedora16_x86_64.dsk
$> glance index

The last command should return a list of the images registered with the Glance image registry.

Downloading Existing Images

If you don't need a functioning Fedora 16 and want the smallest possible images, just download this set of images commonly used by OpenStack developers for testing and register them with Nova:

$> mkdir images
$> cd images
$> curl -L http://github.com/downloads/citrix-openstack/warehouse/tty.tgz | tar xvfzo -
$> glance add name=aki-tty disk_format=aki container_format=aki is_public=true < aki-tty/image
$> glance add name=ami-tty disk_format=ami container_format=ami is_public=true < ami-tty/image
$> glance add name=ari-tty disk_format=ari container_format=ari is_public=true < ari-tty/image

Then to start the image:

$> euca-run-instances ami-tty --kernel aki-tty --ramdisk ari-tty -k mykey

Volumes

If you use the Chrome browser, kill it before embarking on this section, as it has been known to cause the lvcreate command to fail with 'incorrect semaphore state' errors.

Note when setting up volumes in production, make sure you don't put your volume nodes on the same network as your guests when using the default volume driver, as all the iscsi targets are discoverable and accessible without any security.

Start the SCSI target daemon

$> sudo systemctl start tgtd.service
$> sudo systemctl enable tgtd.service

Create a new 1GB volume

$> VOLUME=$(euca-create-volume -s 1 -z nova | awk '{print $2}')

View the status of the new volume, and wait for it to become 'available'

$> watch "euca-describe-volumes | grep $VOLUME | grep available"

Re-run the previously terminated instance if necessary:

$> INSTANCE=$(euca-run-instances f17-jeos -k mykey | grep INSTANCE | awk '{print $2}')

or:

$> INSTANCE=$(euca-run-instances ami-tty --kernel aki-tty --ramdisk ari-tty -k mykey | grep INSTANCE | awk '{print $2}')

Make the storage available to the instance (note -d is the device on the compute node)

$> euca-attach-volume -i $INSTANCE -d /dev/vdc $VOLUME

ssh to the instance and verify that the vdc device is listed in /proc/partitions

$> cat /proc/partitions

Now make the device available if /dev/vdc is not already present

$> mknod /dev/vdc b 252 32

Create and mount a file system directly on the device

$> mkfs.ext3 /dev/vdc
$> mkdir /mnt/nova-volume
$> mount /dev/vdc /mnt/nova-volume

Display some file system details

$> df -h /dev/vdc

Create a temporary file:

$> echo foo > /mnt/nova-volume/bar

Terminate and re-run the instance, then re-attach the volume and re-mount within the instance as above. Your temporary file will have persisted:

$> cat /mnt/nova-volume/bar

Unmount the volume again:

$> umount /mnt/nova-volume

Exit from the ssh session, then detach and delete the volume:

$> euca-detach-volume $VOLUME
$> euca-delete-volume $VOLUME

Floating IPs

You may carve out a block of public IPs and assign them to instances.

First thing you need to do is make sure that nova is configured with the correct public network interface. The default is eth0, but you can change it by e.g.

$> sudo openstack-config-set /etc/nova/nova.conf DEFAULT public_interface em1
$> sudo systemctl restart openstack-nova-network.service

Then you can do e.g.

$> sudo nova-manage floating create 172.31.0.224/28
$> euca-allocate-address
$> euca-associate-address -i i-00000012 172.31.0.224
$> ssh -i nova_key.priv root@172.31.0.224
$> euca-disassociate-address 172.31.0.224
$> euca-release-address 172.31.0.224

VNC access

To setup VNC access to guests through the dashboard:

nova-novncproxy reads some parameters in /etc/nova/nova.conf file. First you need to configure your cloud controller to enable VNC

novncproxy_host = 0.0.0.0
novncproxy_port = 6080

and in the nova compute nodes you need something like this

novncproxy_base_url=http://NOVNCPROXY_FQDN:6080/vnc_auto.html
vnc_enabled=true
vncserver_listen=COMPUTE_FQDN
vncserver_proxyclient_address=COMPUTE_FQDN

You should also make sure that openstack-nova-consoleauth has been started on the controller node:

$ controller> sudo /etc/init.d/openstack-nova-consoleauth restart

After restarting nova services on both nodes the newly created machines will run the qemu-kvm with a parameter -vnc compute_fqdn:display_number. Then after starting the novncproxy and connecting to the dashboard it will discover the host and point to the novncproxy with the appropriate values and connect to the VM.

Note ensure than the iptables entries for VNC ports (5900+DISPLAYNUMBER) are allowed.

Migrate and Resize

This is implemented currently by transferring the images between compute nodes over ssh. Therefore currently you need to make these adjustments on each compute node to allow that.

  • Allow logins for the nova user
 # usermod -s /bin/bash nova
 # su - nova
 $ chcon -u system_u -r object_r -t user_home_t .
 $ mkdir -p -m 700 --context=system_u:object_r:ssh_home_t:s0 .ssh && cd .ssh
  • Disable host identity checking by adding this to ssh config
 $ cat > config <<EOF
 Host * 
   StrictHostKeyChecking no 
   UserKnownHostsFile=/dev/null 
 EOF
  • Generate and distribute ssh key
 $ ssh-keygen -f id_rsa -b 1024 -P ""
 $ scp /var/lib/nova/.ssh/id_rsa.pub root@otherHost:/var/lib/nova/.ssh/authorized_keys
 # chown nova:nova /var/lib/nova/.ssh/authorized_keys

To improve the SELinux config in future the above context manipulations will need to be done centrally, so that restorecon works as expected.

Live Migration of VM instances

First note the official OpenStack docs on the feature and a doc patch distinguishing libvirt live migration.

  • Seting NFS server
    • Make an nfs share with no_root_squash (nova uses root-wrap to chown the instance's disk to qemu:qemu)
    • Make nova user and qemu user:
nova:x:162:162::/home/nova:/bin/bash
qemu:x:107:107::/home/qemu:/bin/bash
    • chown -R nova:nova /the/nfs/share
  • Mount nfs share on each host at /var/lib/nova/instances
  • Configure libvirt
    • See the libvirt wiki as to how to create certificates.
    • Edit /etc/libvirt/libvirt.conf
listen_tcp = 1
tcp_port = "16509"
auth_tcp = "none"
    • Edit /etc/sysconfig/libvirtd
LIBVIRTD_ARGS="--listen"
  • Restart libvirtd & OpenStack compute services

Deployment

Adding a Compute Node

Okay, everything so far has been done on a single node. The next step is to add another node for running VMs.

Let's assume the machine you've set up above is called 'controller' and the new machine is called 'node'.

First, open the qpid, MySQL, Nova API and iSCSI ports on controller:

$ controller> sudo lokkit -p 3306:tcp
$ controller> sudo lokkit -p 5672:tcp
$ controller> sudo lokkit -p 9292:tcp
$ controller> sudo lokkit -p 3260:tcp
$ controller> sudo service libvirtd reload

Then make sure that ntp is enabled on both machines:

$> sudo yum install -y ntp
$> sudo service ntpd start
$> sudo chkconfig ntpd on

Install libvirt and nova on node:

$ node> sudo yum install --enablerepo=epel-testing openstack-nova python-keystone openstack-utils
$ node> sudo service libvirtd start
$ node> sudo chkconfig libvirtd on
$ node> sudo setenforce 0

Configure nova so that node can find the services on controller:

$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT qpid_hostname controller
$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT sql_connection mysql://nova:nova@controller/nova
$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT glance_api_servers controller:9292
$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT iscsi_ip_prefix 172.31.0.107
$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT auth_strategy keystone

(The {{{iscsi_ip_prefix}}} value is the IP address of the controller node)

Configure the Network interfaces The bridge name should match what use used in the nova-manage command on the controller

$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_network_bridge demonetbr0

The device which should be moved onto the bridge (nova will set up this bridge, once it done you can view it with the brctl command

$ node> sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_interface eth0
$ controller> sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_interface eth0
$ node> brctl show

Enable the compute service:

$ node> sudo service openstack-nova-compute start

Now everything should be running as before, except the VMs are launched either on controller or node. You will only be able to ping/ssh to vm's from the controller node.

Manual Setup of MySQL

As of openstack-nova-2011.3-9.el6 and openstack-nova-2011.3-8.fc16, openstack-nova is now set up to use MySQL by default. If you're updating an older installation or prefer to set up MySQL manually instead of using the openstack-nova-db-setup script, this section shows how to do it.

First install and enable MySQL:

$> sudo yum install -y mysql-server
$> sudo service mysqld start
$> sudo chkconfig mysqld on

Set a password for the root account and delete the anonymous accounts:

$> mysql -u root
mysql> update mysql.user set password = password('iamroot') where user = 'root';
mysql> delete from mysql.user where user = '';

Create a database and user account specifically for nova:

mysql> create database nova;
mysql> create user 'nova'@'localhost' identified by 'nova';
mysql> create user 'nova'@'%' identified by 'nova';
mysql> grant all on nova.* to 'nova'@'%';

(If anyone can explain why nova@localhost is required even though the anonymous accounts have been deleted, I'd be very grateful :-)

Then configure nova to use the DB and install the schema:

$> sudo openstack-config-set /etc/nova/nova.conf DEFAULT sql_connection mysql://nova:nova@localhost/nova
$> sudo nova-manage db sync

As a final sanity check:

$> mysql -u nova -p nova
Enter password:
mysql> select * from migrate_version;

Miscellaneous

Smoke Tests

Nova comes with a selection of fairly basic smoke tests which you can run against your installation. It can be useful to use these to sanity check your configuration.

First off, you need the nova-adminclient python library which isn't yet packaged:

$> sudo yum install python-pip
$> sudo pip-python install nova-adminclient

Then you need a user and project both named admin:

$> sudo nova-manage user admin admin
$> sudo nova-manage project create admin admin
$> sudo nova-manage project zipfile admin admin
$> unzip nova.zip
$> . ./novarc

Make sure you have the tty images imported as described above. You also need a block of floating IPs created, also as described above.

Then, run the tests from a fedpkg checkout:

$> fedpkg clone openstack-nova
$> cd openstack-nova
$> fedpkg switch-branch f18
$> fedpkg prep
$> cd nova-2012.2/smoketests
$> python ./run_tests.py

All the tests should pass.

If you run into import errors such as:

ImportError: No module named nose

or:

ImportError (No module named paramiko)

simply install the missing dependency as follows:

$> sudo yum install -y python-nose.noarch
$> sudo yum install -y python-paramiko.noarch

Cleanup

While testing OpenStack, you might want to delete everything related to OpenStack and start testing with a clean slate again.

Here's how. First, make sure to terminate all running instances:

$> euca-terminate-instances ...

Double check that you have no lingering VMs, perhaps saved to disk:

$> virsh list --all && virsh undefine
$> rm -f /var/lib/libvirt/qemu/save/instance-00000*

Then stop all the services:

$> for iii in /usr/lib/systemd/system/openstack-*.service; do sudo systemctl stop $(basename $iii); done

Delete all the packages:

$> sudo yum erase python-glance python-nova* python-keystone* openstack-swift* memcached

Delete the nova and keystone tables from the MySQL DB:

$> mysql -u root -p -e 'drop database nova;'
$> mysql -u root -p -e 'drop database keystone;'

Delete the nova-volumes VG:

$> sudo vgchange -an nova-volumes
$> sudo losetup -d /dev/loop0
$> sudo rm -f /var/lib/nova/nova-volumes.img

Take down the bridge and kill dnsmasq:

$> sudo ip link set demonetbr0 down
$> sudo brctl delbr demonetbr0
$> sudo kill -9 $(cat /var/lib/nova/networks/nova-demonetbr0.pid)

Remove all directories left behind from the packages:

$> sudo rm -rf /etc/{glance,nova,swift,keystone,openstack-dashboard} /var/lib/{glance,nova,swift,keystone} /var/log/{glance,nova,swift,keystone} /var/run/{glance,nova,swift,keystone}

Remove swift storage device (if we don't want the data)

$> sudo umount /srv/node/partitions
$> sudo losetup -d $DEVICE
$> rm /tmp/swiftstorage 

Finally, restart iptables to clear out all rules added by Nova. You also need to reload libvirt's iptables rules:

$> sudo service iptables restart
$> sudo service libvirtd restart