Virtualization short cuts for noobs

Virtualization short cuts for noobs

Introduction

References

http://www.linux-kvm.org/page/Documents

KVM documentation page.

http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge

description of the linux bridge implementation.

http://en.gentoo-wiki.com/wiki/Bridging_Network_Interfaces

http://en.gentoo-wiki.com/wiki/KVM#Networking_2

Shows direct and indirect bridging.

http://blog.loftninjas.org/2008/08/18/using-attach-interface-with-libvirt-and-qemukvm-on-ubuntu/

about attach-interface.

http://tjworld.net/wiki/Linux/Ubuntu/VirtualMachinesWithVDENetworking

Virtual Machines With VDE Networking

http://www.mythicalbeast.co.uk/linux/dnsmasq_howto.html

HOWTO for dnsmasq

http://www.tuxradar.com/content/howto-linux-and-windows-virtualization-kvm-and-qemu

hands-on KVM usage and a bit of background information.

http://www.ibm.com/developerworks/library/l-linuxvirt/index.html

An overview of virtualization methods, architectures, and implementations. From 2006.

Virtualization_Quick_Start

Get going with KVM.

Open issues

how do we prevent container mem swap

It seems that KVM will allow a containers memory to be swapped. This must be prevented.

How to pin to cpus permanently

it seems that editing the /etc/libvirt/qemu/domu.xml doesn't work.

Operating Environment

Which packages you need in order to run virtualization.

yum install @virtualization

or

1. yum install libvirt
2. yum install qemu-kvm
3. yum install python-virtinst
4. yum install virt-manager
5. yum install virt-viewer
6. yum install virt-top
7. service libvirtd restart

Storage

• File based
• LVM Based
• NFS based
• iSCSI

Network

Network configuration needs.

• Bridging
  • Direct
  • NAT
• NAT/Route

Get direct bridging

Looks better: [Guest Configuration on Fedora 12 with Netcf] Source: [KVM w/o dnsmasq on CentOS 5.2 X64].

1. cd /etc/sysconfig/network-scripts
2. mkdir bak
3. cp -p ifcfg-eth0 bak
4. virsh iface-dumpxml eth0 > bak/eth0.xml
5. cp bak/eth0.xml bak/kvmbr0.xml
6. vi bak/kvmbr0.xml
7. virsh iface-define bak/kvmbr0.xml
8. virsh iface-list
9. service network restart
10. virsh edit fc13_1
11. brctl show

The following didn't cut it, it simply prevented the DomU in starting, due to the missing 'default' network...

1. virsh net-destroy default
2. virsh net-autostart --disable default

1. virsh net-autostart default

bak/kvmbr0.xml

<interface type='bridge' name='virbr0'>
  <start mode="onboot"/>
  <protocol family="ipv4">
    <dhcp/>
  </protocol>
  <bridge stp="on">
    <interface type="ethernet" name="eth0">
      <mac address='d8:d3:85:e1:29:68'/>
    </interface>
  </bridge>
</interface>

virsh net-dumpxml default

<network>
  <name>default</name>
  <uuid>4eb269bb-b949-49ae-95b1-6e281b634c47</uuid>
  <forward mode='route'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>

Storage creation

LVM creation

1. Create a partition of type 8e (Linux LVM)
2. pvcreate /dev/sda5
3. vgcreate LX /dev/sda5
4. lvcreate --size 10G --name vol01 LX

Resource allocation

KVM

CPU

Host pinning

You pin the host OS to a CPU set, by telling the kenerl which CPUs to keep away from. So if you want your host OS to only use cpu0 then you must provide the kernel parm: isolcpus=1-3. In case of an 8 core CPU it would be 1-7 etc.

Add it to the end of the kernel entry in '/boot/grub/menu.lst'

Xen

CPU

• http://pkill.info/b/2258/managing-xen-dom0s-cpu-and-memory/

Domain U pinning

Pinning does not mean exclusive access, if you pin Dom1 to cpu1 then Dom1 can only run on cpu1, but it does not prevent other domains to run on cpu1.

If you want a Domain to have exclusive access to a cpu then you must pin all other DomUs to a cpu set that excludes the exclusive cpu. e.g.

 On a 4 CPU system:
If you want Dom1 to have exclusive access to cpu1, then you do the following:

1. Pin all other domains to cpuset 2-3.
2. Pin Dom1 to cpu1.
3. Pin the host domain to cpu0.
   • Or you could have Dom1 on cpu1 and both host and the other domains on the 0,2-3 cpu set.

I/O

• http://pkill.info/b/810/an-io-performance-comparison-between-loopback-backed-and-blktap-backed-xen-file-backed-vbd/

Number of devices

• http://pkill.info/b/223/add-more-loop-device-on-linux/

Administration

virsh

virsh help lists the supported commands.

See also [28. Managing guests with virsh]

Virsh operations commands of interest

console

get access to the console in the DomU.

destroy

as if you pulled the power cord, preferably use 'shutdown'.

reboot

reboot the domain.

shutdown

shutdown the domain.

undefine

remove all references to the domain. (Note; not sure if it is actually *all*).

dumpxml

list

List the active domains. Use '--all' to include the the inactive containers to the list.

Virsh network commands of interest

domifstat

interface statistics.

• virsh domifstat z3 vif7.0

attach-interface

attach a network interface.

• virsh attach-interface z3 bridge eth1 --target eth1 --mac xx:xx:xx:xx:xx:xx

net-edit

For an description of the xml structure see: [XML format]

net-list

list networks.

virsh cpu commands of interest

vcpuping <domain-id> <vcpuid> <cpulist>

Pin a domains virtual cpu to a list of cpu's.

• • virsh vcpupin z2 0 2

1. virsh vcpuinfo z2

VCPU: 0 CPU: 2 State: idle CPU time: 9.8s CPU Affinity: --y-----

virt-mananger

Access guest files

virt-cat

virt-cat Guest /var/log/messages # or /var/log/dmesg etc

guestfish

guestfish --ro -i -d Guest

• • ><fs> less /var/log/messages
  • guestfish -i -d Guest

virt-edit

virt-edit Guest /boot/grub/grub.conf

><fs> emacs /boot/grub/grub.conf

virt-rescue

virt-rescue Guest

><rescue> mount /dev/vg_foo/lv_root /sysroot
><rescue> mount /dev/vda1 /sysroot/boot
><rescue> vi /sysroot/boot/grub/grub.conf

Creating DomUs

Creating a CentOS 5.5 DomU

Xen: source:NFS, storage:LVM network:bridge;4

virt-install -p --name=z4 --ram=512 --vcpus=1 --disk path=/dev/loom/z4 --os-type=linux --os-variant=rhel5 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/anaconda-ks.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"

KVM: source:NFS, storage:LVM network:bridged-4

virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --network=bridge:virbr0 --network=bridge:virbr1 --network=bridge:virbr2 --network=bridge:virbr3 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/kvm_centos55.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"

Trouble shooting

virsh ops

error: Timed out during operation: cannot acquire state change lock

# virsh destroy z02
error: Failed to destroy domain z02
error: Timed out during operation: cannot acquire state change lock

• The system had just been rebooted.
  • The z01 and z02 was active, when the reboot command was executed on the host.
• when the host came back up, both z01 and z02 was paused.
• virt-top, cleared the screen, and then didn't show anything.
• I could stop z01 with virsh destroy z01
• There was a qemu-kvm process running, and doing a kill on that finished off z02

See also: http://us.generation-nt.com/answer/bug-602715-libvirt-bin-locking-problem-libvirt-help-201086371.html

virt-install

error: unable to connect to '/var/run/libvirt/libvirt-sock', libvirtd may need to be started: No such file or directory

run service libvirtd restart

ERROR Host does not support virtualization type 'xen'

running on KVM, the '-p | --para-virt' is not supported.

error: Failed to get local hostname

• The hostname must be set.
• The system must be able to do a dns lookup of the hostname.
  • e.g. put the hostname in /etc/hosts
  • Re: et-mgmt-tools virsh failed to get local hostname

virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/anaconda-ks.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"


Starting install...
Retrieving file .treeinfo...                                                     |  834 B     00:00 ... 
Retrieving file vmlinuz...                                                       | 3.7 MB     00:00 ... 
Retrieving file initrd.img...                                                    |  15 MB     00:00 ... 
Creating domain...                                                               |    0 B     00:00     
error: Failed to get local hostname

Domain installation still in progress. You can reconnect to 
the console to complete the installation process.

Kickstart issues

That directory could not be mounted from the server

A1: Wrong IP address used for NFS server. A2: The path was wrong.

• tail /var/log/messages:
  • refused mount request from 192.168.42.31 for /export/isos/centos55 (/): no export entry