FedoraCore5XenQuickstart

About Xen
The Xen virtual machine system is now available in Fedora Core 4 and above versions. More information on Xen itself can be found at http://wiki.xensource.com/xenwiki/ and the Fedora  Xen  page.

Fedora is following the -unstable (3.0.x) Xen development line at the moment, which does occasionally lead to things being broken but also lets us track a lot of the more interesting work going on there. It also prevents the current NetBSD/xen, FreeBSD 5.3 and Solaris 11beta ports from running because they are all based on Xen 2.0.x.

About This Quick Start
Since setting up to run Xen isn't entirely straight-forward, here's a run-through of what should work for configuring a Fedora Core 5 system as a Xen host, and setting up a single Xen guest.

This Quick Start is an update version of the Fedora Core 4 Xen Quick Start  page. A number of things have changed since FC4 and Xen 2 such as package names and configuration options, this guide is aimed at being a fresh start rather than an upgrade guide.

System Requirements

 * Your system must use GRUB, the default boot loader for Fedora [[FootNote(This is required because you actually boot the Xen hypervisor and it then starts the Linux kernel. It does this using the Multi .  The Xen hypervisor itself requires ~32 MB. Plus the memory required for your guest systems.)]
 * Sufficient storage space for the guest operating systems. A minimal command-line Fedora system requires around 600MB of storage, a standard desktop Fedora system requires around 3GB.
 * SELinux must not be enforcing it's policy, or some of the operations below will fail. Use  to set SELinux to not enforce, and reboot to let this change take effect.

Conventions
Commands which require root privileges are prefixed with the character '#'. To become root, issue the command 'su' as a normal user and supply the root password. Alternatively, use another system such as 'sudo' if you have it setup.

Installing the Xen Software
Before we start we need to set up the basics of the Xen system. To install the main software, type the command:

The  package contains the Xen hypervisor and tools needed to run virtual machines on x86 systems;   is the Xen-enabled host system kernel  and   is the Xen guest system kernel.

Once this is done, you should have an entry set up in the file  for booting the   kernel. The  kernel is not set as the default boot option.

To set GRUB to boot with  by default, edit   and set the default to the xen-hypervisor kernel: default=2

This is an example, with the   option to start the host system on runlevel 3:

default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Fedora Core (2.6.15-1.29_FC5hypervisor) root (hd0,0) kernel /xen.gz-2.6.15-1.29_FC5 module /vmlinuz-2.6.15-1.29_FC5hypervisor ro root=/dev/VolGroup01/LogVol00 init 3 module /initrd-2.6.15-1.29_FC5hypervisor.img

Enabling Xen
Once you've rebooted with the  GRUB option, you should be running with a Xen-enabled kernel [[FootNote(To check that you are running a   kernel use ' ' - the name of the kernel should contain  )].

Now start the Xen service:

To check that Xen is running successfully, enter the following command:

You should see Domain-0 (the Xen-enabled host) running: [root@xen-hypervisor ~] # xm list Name                             ID Mem(MiB) VCPUs State  Time(s) Domain-0                          0      940     1 r-   267.8

To enable the Xen service to automatically start on bootup, enter the following:

Creating A Guest System
These instructions set up a simple base Fedora installation as a Xen guest system.

Create a Virtual Disk for the Guest System
{{Admon/warning | This example system uses the kernel from the  package, as the kernel for your guest domain must currently be kept outside of the guest itself.

First, create an image file to use as the storage for our Fedora guest system. This acts as a virtual hard drive for the guest system. This command creates an image file with a size of 1 GB. To create a larger filesystem increase the  value. dd if=/dev/zero of=fedora-guest1.img bs=1M count=1 seek=1024

Now, format the image file by creating an ext3 filesystem on it. /sbin/mke2fs -F -j fedora-guest1.img

You should now be able to mount the image on a temporary mountpoint, say

Setup the Base Environment
To install a guest system on to the image we need to set up some basic files on the filesystem of the image that have to be different for Xen (and for the system installation with ).

First up create some diectories that we will need in the following steps:

Create some required device nodes in the image. [[FootNote(Using an initrd and thus udev seems to work as well with a Fedora Core development tree as of 2005-09-16. Just run depmod and mkinitrd from xen-hypervisor for the xen-guest kernel; make sure to install a xen-guest kernel into the guest system; perform a special depmod run as this is still missing from the kernel rpm.)] :

Create the guests  file inside the image (i.e. /mnt/etc/fstab), something simple like the following should work: /dev/sda1              /                       ext3    defaults 1 1 none                   /dev/pts                devpts  gid=5,mode=620 0 0 none                   /dev/shm                tmpfs   defaults 0 0 none                   /proc                   proc    defaults 0 0 none                   /sys                    sysfs   defaults 0 0

Finally, make sure /sys/ is mounted inside the image so that mkinitrd won't fail

Configure YUM
We need to set up a temporary configuration for yum so we can install our guest system. Create a yum-install.conf file that we will use to find our repositories, this file can be created in a temporary location as it will not be needed after we successfully install our guest.

For a stable release the contents of the yum-install.conf file should contain: [main] cachedir=/var/cache/yum debuglevel=2 logfile=/var/log/yum.log exclude=*-debuginfo gpgcheck=0 obsoletes=1 reposdir=/dev/null

[base] name=Fedora Core 5 - $basearch - Base mirrorlist=http://fedora.redhat.com/download/mirrors/fedora-core-5 enabled=1

[updates-released] name=Fedora Core 5 - $basearch - Released Updates mirrorlist=http://fedora.redhat.com/download/mirrors/updates-released-fc5 enabled=1

For a rawhide development system the contents of the yum-install.conf file should instead contain: [main] cachedir=/var/cache/yum debuglevel=2 logfile=/var/log/yum.log exclude=*-debuginfo gpgcheck=0 obsoletes=1 reposdir=/dev/null

[development] name=Fedora Core Development Tree mirrorlist=http://fedora.redhat.com/download/mirrors/fedora-core-rawhide enabled=1

If you want to use a local Fedora mirror, then you'll want to set the  parameter and comment out the   parameter before using it; otherwise, it will point to the default set of mirror Websites.

Install a Base Operating System
Firstly we need to install the  package into the image. This will pull in dependencies that total to a bit over 60 MiB.

After the the  package and dependencies have installed you should update the initrd:

Then, decide which package group(s) you want to install. Start with Core and Base (or for the space constrained just Core, however this is more difficult as you will be missing utilities such as yum).

Install the guest system with the package groups by running. For example, to install with Core and Base use the command:

Grab a coffee and have a snack, it's going to take a little while depending on your connection speed to the yum repository :-)

Come back and if everything went okay, you'll have a minimal install in.

Do any other configuration you want to on the file system and then unmount it:

Registering a Guest System as a Domain
Now, we just have to create a configuration file in. As a simple example, the following configuration file is :

kernel ="/boot/vmlinuz-2.6.15-1.29_FC5guest" memory = 64 name = "fedora-guest1" vif = [''] disk = root = "/dev/sda1" extra = "ro selinux=0 3"

Substitute  with the xen-guest kernel you installed. Additionally, the device sda1 listed here as the disk is related to the  you created above.

Before actually creating the guest domain you may want to decrease the amount of memory being allocated for your host system (domain0), in order to leave adequate room in RAM for your guest domains. This is especially important as the Xen system won't page your guest domains to your host domain's swap space. Constraining the RAM space used by your host domain can help assure that adequate RAM is available for your guests.

The way to do this is to run the following commands as root:

xm mem-max 0 384 xm mem-set 0 384

The mem-max command restricts the host domain (the 0 parameter signifies domain 0) from ever increasing its memory usage beyond 384 megabytes. The mem-set command forces the host domain to immediately shrink its memory allocation down to 384 megabytes.

It may take a bit of time for the domain 0 kernel to page out memory over the new limit, so you should give it a minute or so to complete.

When this is done, your system should have enough RAM free to allow you allocate one or more pretty good sized guest domains.

Now, create a new domain:

xm create -c fedora-guest1

You should now see the ever familiar boot process, at the end you should see the login prompt at which point you can login as 'root' and begin playing around.

If you want to disconnect from the running Xen guest console, this can be done with the key control key combination: ^

To reconnect to any running xen guest console type: xm console fedora-guest1

To restart the guest domain can be done by connecting to the host domain with another window and running the following: xm shutdown fedora-guest1 xm create -c fedora-guest1

This is pretty early and rough, but it's enough to starting playing with. The next step for development is getting to where you can do actual installations in a Xen guest environment and then being able to boot kernels which are on the guest's file system.

Automatically Loading Domains On Startup
If you want your domains to automatically start when rebooting the system make sure that you enable the  service, and place the Xen configuration file for the domains in the   directory.

To enable the  service:

Note that the /etc/init.d/xendomains script, as of RPM xen-3.0-0.20050912.fc4, seems to be broken with respect to its use of the LSB init functions. This is arguably due to a bug in Fedora's LSB support, and has been filed in Fedora's bugzilla as bug 171052 and bug 171056. The latter bug includes a modified version of the /etc/init.d/xendomains script as an attachment which you can use instead.

Troubleshooting
If you're running into issues with your Xen installation there are two log files on the host system to help you out. The file  holds the same information as you receive when running ' ', and contains basic information. Unfortunately these log messages are often very short and contain little useful information. The following is the output of trying to create a domain running the kernel for NetBSD/xen.

[2005-06-27 02:23:02 xend] ERROR (SrvBase:163) op=create: Error creating domain:(0, 'Error') Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvBase.py", line 107, in _perform val = op_method(op, req) File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py", line 71, in op_create raise XendError("Error creating domain: " + str(ex)) XendError: Error creating domain: (0, 'Error')

The second file,  usually contains much more detailed information. Trying to start the NetBSD/xen kernel will result in the following log output:

ERROR: Will only load images built for Xen v3.0 ERROR: Actually saw: 'GUEST_OS=netbsd,GUEST_VER=2.0,XEN_VER=2.0,LOADER=generic,BSD_SYMTAB' ERROR: Error constructing guest OS

If you are reporting errors or need help with your Xen setup you should always include both the  and   messages that you receive.

Q: I am trying to start then xend service and nothing happens, then when I do a  I get the following: Error: Error connecting to xend: Connection refused. Is xend running?

Alternatively, I run  manually and get the following error: ERROR: Could not obtain handle on privileged command interface (2 = No such file or directory) Traceback (most recent call last): File "/usr/sbin/xend", line 33, in ? from xen.xend.server import SrvDaemon File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDaemon.py", line 21, in ? import relocate File "/usr/lib/python2.4/site-packages/xen/xend/server/relocate.py", line 26, in ? from xen.xend import XendDomain File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 33, in ? import XendDomainInfo File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 37, in ? import image File "/usr/lib/python2.4/site-packages/xen/xend/image.py", line 30, in ? xc = xen.lowlevel.xc.xc RuntimeError: (2, 'No such file or directory')

A: You have rebooted your host into a kernel that is not a xen-hypervisor kernel. Yes I even did this myself in testing :)

You either need to select the xen-hypervisor kernel in the grub menu at boot time or set the xen-hypervisor kernel as default in your grub.conf file.

Getting Help
If the Troubleshooting section above does not help you to solve your problem, check the Red Hat Bugzilla for existing bug reports on Xen in FC5. The product is "Fedora Core", and the component is "Xen". These reports contain useful advice from fellow xen testers and often describe work-arounds.

Discussion on Fedora Xen support issues occur on the Fedora Xen mailing list

The Xen 3 User Manual can be found at http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html

For general Xen issues and useful information check the Xen project documentation, and mailing list archives.