Virtualization short cuts for noobs

From FedoraProject

Jump to: navigation, search


Virtualization short cuts for noobs




Acronyms, descriptions

KVM documentation page.
description of the linux bridge implementation.
Shows direct and indirect bridging.
about attach-interface.
Virtual Machines With VDE Networking
HOWTO for dnsmasq
hands-on KVM usage and a bit of background information.
An overview of virtualization methods, architectures, and implementations. From 2006.
Get going with KVM.

Open issues

how do we prevent container mem swap
It seems that KVM will allow a containers memory to be swapped. This must be prevented.
How to pin to cpus permanently
it seems that editing the /etc/libvirt/qemu/domu.xml doesn't work.

Operating Environment

Which packages you need in order to run virtualization.

yum install @virtualization


  1. yum install libvirt
  2. yum install qemu-kvm
  3. yum install python-virtinst
  4. yum install virt-manager
  5. yum install virt-viewer
  6. yum install virt-top
  7. service libvirtd restart


  • File based
  • LVM Based
  • NFS based
  • iSCSI


Network configuration needs.

  • Bridging
    • Direct
    • NAT
  • NAT/Route

Get direct bridging

Looks better: [Guest Configuration on Fedora 12 with Netcf] Source: [KVM w/o dnsmasq on CentOS 5.2 X64].

  1. cd /etc/sysconfig/network-scripts
  2. mkdir bak
  3. cp -p ifcfg-eth0 bak
  4. virsh iface-dumpxml eth0 > bak/eth0.xml
  5. cp bak/eth0.xml bak/kvmbr0.xml
  6. vi bak/kvmbr0.xml
  7. virsh iface-define bak/kvmbr0.xml
  8. virsh iface-list
  9. service network restart
  10. virsh edit fc13_1
  11. brctl show

The following didn't cut it, it simply prevented the DomU in starting, due to the missing 'default' network...

  1. virsh net-destroy default
  2. virsh net-autostart --disable default

  1. virsh net-autostart default


<interface type='bridge' name='virbr0'>
  <start mode="onboot"/>
  <protocol family="ipv4">
  <bridge stp="on">
    <interface type="ethernet" name="eth0">
      <mac address='d8:d3:85:e1:29:68'/>

virsh net-dumpxml default

  <forward mode='route'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <ip address='' netmask=''>
      <range start='' end='' />

Storage creation

LVM creation

  1. Create a partition of type 8e (Linux LVM)
  2. pvcreate /dev/sda5
  3. vgcreate LX /dev/sda5
  4. lvcreate --size 10G --name vol01 LX

Resource allocation



Host pinning

You pin the host OS to a CPU set, by telling the kenerl which CPUs to keep away from. So if you want your host OS to only use cpu0 then you must provide the kernel parm: isolcpus=1-3. In case of an 8 core CPU it would be 1-7 etc.

Add it to the end of the kernel entry in '/boot/grub/menu.lst'



Domain U pinning
Pinning does not mean exclusive access, if you pin Dom1 to cpu1 then Dom1 can only run on cpu1, but it does not prevent other domains to run on cpu1.

If you want a Domain to have exclusive access to a cpu then you must pin all other DomUs to a cpu set that excludes the exclusive cpu. e.g.

 On a 4 CPU system:
If you want Dom1 to have exclusive access to cpu1, then you do the following:
  1. Pin all other domains to cpuset 2-3.
  2. Pin Dom1 to cpu1.
  3. Pin the host domain to cpu0.
    • Or you could have Dom1 on cpu1 and both host and the other domains on the 0,2-3 cpu set.


Number of devices



virsh help lists the supported commands.

See also [28. Managing guests with virsh]

Virsh operations commands of interest

get access to the console in the DomU.
as if you pulled the power cord, preferably use 'shutdown'.
reboot the domain.
shutdown the domain.
remove all references to the domain. (Note; not sure if it is actually *all*).

List the active domains. Use '--all' to include the the inactive containers to the list.

Virsh network commands of interest

interface statistics.
  • virsh domifstat z3 vif7.0
attach a network interface.
  • virsh attach-interface z3 bridge eth1 --target eth1 --mac xx:xx:xx:xx:xx:xx
For an description of the xml structure see: [XML format]
list networks.

virsh cpu commands of interest

vcpuping <domain-id> <vcpuid> <cpulist>
Pin a domains virtual cpu to a list of cpu's.
    • virsh vcpupin z2 0 2
  1. virsh vcpuinfo z2

VCPU: 0 CPU: 2 State: idle CPU time: 9.8s CPU Affinity: --y-----



remote management

Access guest files

virt-cat Guest /var/log/messages # or /var/log/dmesg etc
guestfish --ro -i -d Guest
    • ><fs> less /var/log/messages
    • guestfish -i -d Guest
virt-edit Guest /boot/grub/grub.conf
><fs> emacs /boot/grub/grub.conf
virt-rescue Guest
><rescue> mount /dev/vg_foo/lv_root /sysroot
><rescue> mount /dev/vda1 /sysroot/boot
><rescue> vi /sysroot/boot/grub/grub.conf

Creating DomUs

Creating a CentOS 5.5 DomU

Xen: source:NFS, storage:LVM network:bridge;4

virt-install -p --name=z4 --ram=512 --vcpus=1 --disk path=/dev/loom/z4 --os-type=linux --os-variant=rhel5 --location=nfs: -x "ks=nfs: ip= netmask= gateway= dns="

KVM: source:NFS, storage:LVM network:bridged-4

virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --network=bridge:virbr0 --network=bridge:virbr1 --network=bridge:virbr2 --network=bridge:virbr3 --location=nfs: -x "ks=nfs: ip= netmask= gateway= dns="

Trouble shooting

virsh ops

error: Timed out during operation: cannot acquire state change lock

# virsh destroy z02
error: Failed to destroy domain z02
error: Timed out during operation: cannot acquire state change lock
  • The system had just been rebooted.
    • The z01 and z02 was active, when the reboot command was executed on the host.
  • when the host came back up, both z01 and z02 was paused.
  • virt-top, cleared the screen, and then didn't show anything.
  • I could stop z01 with virsh destroy z01
  • There was a qemu-kvm process running, and doing a kill on that finished off z02

See also:


error: unable to connect to '/var/run/libvirt/libvirt-sock', libvirtd may need to be started: No such file or directory

run service libvirtd restart

ERROR Host does not support virtualization type 'xen'

running on KVM, the '-p | --para-virt' is not supported.

error: Failed to get local hostname

virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --location=nfs: -x "ks=nfs: ip= netmask= gateway= dns="

Starting install...
Retrieving file .treeinfo...                                                     |  834 B     00:00 ... 
Retrieving file vmlinuz...                                                       | 3.7 MB     00:00 ... 
Retrieving file initrd.img...                                                    |  15 MB     00:00 ... 
Creating domain...                                                               |    0 B     00:00     
error: Failed to get local hostname

Domain installation still in progress. You can reconnect to 
the console to complete the installation process.

Kickstart issues

That directory could not be mounted from the server

A1: Wrong IP address used for NFS server. A2: The path was wrong.

  • tail /var/log/messages:
    • refused mount request from for /export/isos/centos55 (/): no export entry